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@ A process for transferring data via DMA between 
a system resource and a controller via switching 
logic. During a setup write transaction, the switching 
logic is set up to enable DMA data to be transferred 
between a particular system memory and a selected 
system resource. The setup write transaction also is 
used to initialize the DMA byte counter. During a 
subsequent write fransaction, DMA pointer registers 
are initialized with appropriate starting addresses. 
The controller then transmits a DMA start code and 
the system resource responds by transmitting an 
acknowledge code. At that time. DMA data is trans- 
mitted between the controller and the system re- 
source via the switching logic. 
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PROTOCOL FOR TRANSFER OF DMA DATA 



BACKGROUND OF THE INVENTION 

The invention relates to the transfer of data via 
DMA between a system resource and a controller 
via switching logic. 

In connputer systems, blocks of data can be 
quickly transferred between a system resource and 
a system memory via DMA using a controller. 
Before the DMA transfer can begin, certain in- 
formation must be provided for use by the system 
resource and the controller. 

In general, the starting addresses that will be 
accessed by the system resource and by the 
memory are stored in pointer registers. The ad-, 
dresses stored in the pointer registers are incre- 
mented or decremented by the system resource 
and by the controller to access the appropriate 
storage locations. As a result, an entire block of 
data can be transferred without the intervention of 
the main processors in the computer system. 

A byte or word count also is provided and is 
stored in a counter before the DMA transfer begins. 
The count changes as each byte or word of DMA 
data is transferred. 

When the counter determines that all of the 
bytes or words In the DMA data block have been 
transferred, the DMA operation Is complete. 

In many computer systems, multiple system 
resources are present that can perform DMA transr 
fers with a system memory. In these systems, 
switching logic is necessary in order to couple a 
particular system memory to a selected system, 
resource. 

~ However, in systems containing switching log- 
ic, it may be difficult to set up the required data 
path rapidly for an imminent DMA data transfer. 
The difficulty increases if the switching logic must 
be configured to allow DMA data to be transferred 
in a either of tyvo directions between the selected 
system resource and the system memory. 

Therefore, there is a need for a computer sys- 
tem in which the switching logic is efflcieritly set up 
before a DMA transfer starts to enable DMA data to 
be transferred between a particular system mem- 
ory and a selected system resource. There also is 
a need for a computer system in which a simple 
protocol is used to rapidly Initialize the pointer 
registers, counters, and switching logic in prepara- 
tion for a DMA transfer. 



SUMMARY OF THE INVENTION 



It is an object of the invention to overcome at 
least some of the problems associated with the 
prior art. 

It is therefore desirable to provide a process for 
6 efficiently configuring switching logic in order to 
transfer data via DMA between two nodes in a 
network. It is also beneficial to provide a process 
for rapidly initializing the pointer registers and 
counters. 

70 Additional objects and advantages of the inven- 
tion will be set forth in part in the description which 
follows, and in part will be obvious fronr) the de- 
scription, or may be learned by practice of , the 
invention. The objects and advantages of the inven- 

76 tion may be realized and obtained by means of the 
instrumentalities and combinations . particularly 
pointed out in the appended claims. 

To achieve the objects and in accordance with 
the purposes of the invention, as embodied and 

20 broadly described herein, a process is provided for 
transferring data via DMA between components in 
a computer system, wherein the computer system 
includes , a memory controller, a first system re- 
source bus coupled to a system resource, and a 

26 data router. The data router includes switching log- 
ic for coupling the memory controller to the first 
system resource bus. The process includes the 
step of transmitting DMA setup information des- 
ignating a setup write transaction, from the memory 

30 controller, to the system resource, via the data 
router and via the first system resource bus coup- 
led to the system resource. The DMA. setup In- 
formation corresponds to a DMA transfer, and in- - 
dicates a direction of the DMA transfer as either an 

35 up direction from the system resource to the mem- 
ory controller or a down direction from the memory 
controller to the system resource, indicates wheth- 
er the DMA transfer will involve the first system 
resource bus, and indicates whether the DMA 

40 transfer will involve a system resource coupled to 
the first system resource bus. The process also 
includes the steps of decoding the DMA setup 
information, in the data router, during the setup 
write transaction, to determine the direction of the 

45 DMA transfer, to determine whether the DMA trans- 
fer will Involve the first system resource bus, and to ^ 
determine whether the DMA transfer will involve a 
system resource coupled to the first system re- 
source bus; transmitting a start DMA code, from 

60 the memory controller, to the system resource, via 
the data router and via the first system resource 
bus coupled to the system resource; and configur- 
ing the switching logic in the data router, in re- 
sponse to transmission of the DMA start code, and 
in accordance with the DMA setup infonmation de- 
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coded by the data router. The switching logic is 
configured to forward DMA data, from the first 
system resource bus to the memory controller, at 
times when the DMA transfer is in the up direction 
and the DMA transfer will involve the first system 
resource bus and the DMA transfer will Involve a 
system resource coupled to the first system re- 
source bus; and the switching logic is configured to 
fon^ard DMA data, from the memory controller, to 
the first system resource bus, at times when the 
DMA transfer is In the down direction and the DMA 
transfer will involve the first system resource bus 
and the DMA transfer will involve a system re- 
source coupled to the first system resource bus. 
The process further includes the steps^of transmit- 
ting an acknowledge code, from the system re- 
source, to the memory controller, via the first sys- 
tem resource bus coupled to the system resource, 
and via the switching logic In the data router, In 
response to transmission of the DMA start code; 
transmitting DMA data, after transmission of the 
acknowledge code, at times when the DMA transfer 
is in the up direction, from the system resource, to 
the memory controller, via the first system re- 
source bus coupled to the system resource, and 
via the switching logic in the data router; transmit- 
ting DMA data, after transmission of the acknowl- 
edge code, at times when the DMA transfer is in 
the down direction, from the memory controller, to 
the system resource, via the switching logic in the 
data router, and via the first system resource bus 
coupled to the system resource; and transmitting a 
done code, after transmission of DMA data, from 
the system resource, to the memory controller, via 
the first system resource bus coupled to the sys- 
tem resource, and via the switching logic in the 
data router, to indicate successful DMA data trans- 
mission. 

The accompanying drawings, which are incor- 
porated in and constitute a part of this specifica- 
tion, illustrate preferred embodiments of the inven- 
tion and, together with the description of the inven- 
tion, explain the principles of the invention. 



BRIEF DESCRIPTION OF THE DRAWINGS 



Rg. 1 is a block diagram of a preferred embodi- 
ment of fault tolerant computer system which 
practices the present invention; 
Fig. 2 is an illustration of the physical hardware 
containing the fault tolerant computer system in 
Fig. 1; 

Fig. 3 is a block diagram of the CPU module 
shown In the fault tolerant computer system 
shown in Rg. 1 ; 

Rg. 4 is a block diagram of an interconnected 



CPU module and I/O module for the computer 
system shown in .Rg. 1; 

Rg. 5 is a block diagram of a memory module 
for the fault tolerant computer system shown in 
5 Rg. 1; 

Rg. 6 is a detailed diagram of the elements of 
the control logic in the memory module shown 
in Fig. 5; 

Fig. 7 Is a block diagram of portions of the 
10 primary memory controller of the CPU module 

shown in Fig. 3; 

Fig. 8 is a block diagram of the DMA engine in 
the primary memory controller of the CPU mod- 
ule of Fig. 3; 

75 Fig. 9 is a diagram of error processing circuitry 
in the primary memory controller of the CPU 
module of Fig. 3; 

Fig. 10 is a drawing of some of the registers of 
the cross-link in the CPU module shown in Rg. 
20 3: 

Fig. 11 is a block diagram of the elements which 
route control signals in the cross-links of the 
GPU module shown In Fig. 3; 
Fig. 12 Is a block diagram of the elements which 
25 route data and -address signals in the primary 
cross-link of the CPU module shown in Fig. 3; 
Fig. 13 is a state diagram showing the states for 
the cross-link of the CPU module shown in Fig. 
3; 

30 Fig, 14 is a block diagram of the timing system 
for the fault tolerant computer system of Rg. 1 ; 
Rg. 15 is a timing diagram for the clock signals 
generated by the timing system In Fig. 14; ' 
Fig, 16 is a detailed diagram of a phase detector 

35 for the timing system shown in Fig. 14; 

Fig. 17 is a block diagram of an I/O module for 
the computer system of Fig. 1 ; 
Fig. 18 is a block diagram of the firewall ele- 
ment in the I/O module shown in Rg. 17; 

40 Fig. 19 is a detailed diagram of the elements of . 
the cross-link pathway for the computer system 
of- Fig. 1; 

Figs. 20A-20E are data flow diagrams for the 
computer system in Fig, 1 ; 
45 Rg. 21 is a block diagram of zone 20 showing 
the routing of reset signals; 
Fig. 22 is a block diagram of the components 
involved in resets in the CPU module shown in 
Rg. 3; and 

60 Fig. 23 is a diagram of clock reset circuitry. 

Figs. 24A-E are timing diagrams for data trans- 
fers via DMA. 

Figs. 25A-D are timing diagrams for data trans- 
fers via read and write cycles. 

55 

DESCRIPTION OF THE PREFERRED EMBODI- 
MENT 
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Reference will now be made in detail to a 
presently preferred embodiment of the invention, 
an example of which is Illustrated In the accom- 
panying drawings. 



A. SYSTEM DESCRIPTION 

Fig. 1 is a block diagram of a fault tolerant 
computer system. 10 in accordance with the 
present invention. Fault tolerant computer system 

10 includes duplicate systems, called zones, in the 
normal mode, the two zones 11 and 11 ' operate 
simultaneously. The duplication ensures that there 
is no single point of failure and that a single error 
or fault in one of the zones 11 or 11 ' will not 
disable computer system 10, Furthermore, ail such 
faults can be corrected by disabling or ignoring the 
device or element which caused the fault. Zones 

11 and 1l' are shown In Fig. 1 as respectively 
including duplicate processing systems 20 and 20 . 
The duality, however, goes beyond the processing 
system. 

Fig, 2 contains an illustration of the physical 
hardware of fault tolerant computer system 10 and 
graphically Illustrates the duplication of the sys- 
tems. Each zone 11 and 11 is housed in a dif- 
ferent cabinet 12 and 12', respectively. Cabinet 12 
includes battery 13, power regulator 14, cooling 
fans 16, and AC input 17. Cabinet 12' includes 
separate elements corresponding to elements 13, 
14. 16 and 17 of cabinet 12. 

As explained in greater detail below, process- 
ing systems 20 and 20' Include several modules 
interconnected by backplanes. If a module contains 
a fault or error, that module may be removed and 
replaced without disabling computing system 10. 
This is because processing systems 20 and 20' 
are physically separate, have separate backplanes 
into which the modules are plugged, and can op- 
erate independently of each other. Thus modules 
can be removed from and plugged into the back- 
plane of one processing system while the other 
processing system continues to operate. 

In the preferred embodiment, the duplicate pro- 
cessing systems 20 and 20' are identical and con^ 
tain identical modules. Thus, only processing sys- 
tem 20 will be described completely with the un- 
derstanding that processing system 20' operates 
equivalently. 

Processing system 20 Includes CPU module 
30 which is shown in greater detail In Figs. 3 and 4. 
CPU module 30 is interconnected with CPU mod- 
ule 30' In processing system 20' by a cross-link 
pathway 25 which is described in greater detail 
below! Cross-link pathway 25 provides data trans- 
mission paths between processing systems 20 and 
20' and can-les timing signals to ensure that pro- 



cessing systems 20 and 20 operate synchronously 
Processing system 20 also includes 1/0 mod- 
ules 100, 110, and 120. 1/0 modules 100. 110, 120. 
lOO', 110' and 120' are independent devices. I/O 

5 module 100 is shown in greater detail in Rgs. 1, 4, 
and 17. Although multiple I/O modules are shown, 
duplication of such modules is not a requirement of 
the system. Without such duplication, however, 
some degree of fault tolerance will be lost. 

70 Each of the I/O modules 100, 110 and 120 is 
connected to CPU module 30 by dual rail module 
Interconnects 130 and 132. Module interconnects 
130 and 132 serve as the I/O interconnect and are 
routed across the backplane for processing system 

75 20. For purposes of this application, the data path- 
way including CPU 40. memory controller 70, 
cross-link 90 and module interconnect 130 Is con- 
sidered as one rail, and the data pathway including 
CPU 50, memory controller 75, cross-link 95, and 

20 module interconnect 132 is considered as another 
rail. During proper operation, the data on both rails 
is the same. 



26 B. FAULT TOLERANT SYSTEM PHILOSOPHY 

Fault tolerant computer system 1 0* does not 
have a- single point of failure because each element 
Is duplicated. Processing systems 20 and 20' are 

30 each a fail stop processing system which means 
that those systems can detect faults or errors in the 
subsystems and prevent uncontrolled propagation 
of such faults and errors to other subsystems, but 
they have a single point of failure because the 

36 elements In each processing system are not du- 
plicated. 

^ The two fail stop processing systems 20 and 
20 are interconnected by certain elements operat- 
ing in a defined manner to form a fail safe system. 

40 In the fail safe system embodied as fault tolerant 
computer system 10, the entire computer system 
can continue processing even if one of the fail stop 
processing systems 20 and 20' Is faulting. 
^ The two fail stop processing systems 20 and 

45 20' are considered to operate in lockstep synchro- 
nism because CPUs 40, 50, 40' and 50' operate in 
such synchronism. There are three significant ex- 
ceptions. The first is at initialization when a boot- 
strapping technique brings both processors into 

60 synchronism. The second exception is when the 
processing systems 20 and 20' operate indepen- 
dently (asynchronously) on two different workloads. 
The third exception occurs den certain errors arise 
in processing systems 20 and 20'. In this last 

55 exception, the CPU and memory elements in one 
of the processing systems is disabled, thereby 
ending synchronous operation. 

When the system is running in lockstep I/O. 
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only one 1/0 device Is being accessed at any one 
time. All four CPUs 40, 50. 40' and 50', however, 
would receive the same data from that I/O device 
at substantially the same time. In the following 
discussion, It will be understood that lockstep syn- 
chronization of processing systems means that 
only one I/O module is being accessed. 

The synchronism of duplicate processing sys- 
tems 20 and 20' Is Implemented by treating each 
system as a deterministic machine which, starting 
in the same known state and upon receipt of the 
same inputs, will always enter the same machine 
states and produce the same results in the ab- 
sence of error. Processing systems 20 and 20 are 
configured identically, receive the same inputs, and 
therefore pass through the same states. Thus, as 
long as both processors operate synchronously, 
they should produce the same results and enter 
the same state. If the processing systems are not 
in the same state or produce different results, it is 
assumed that one of the processing systems 20 
and 2o' has faulted. The source of the fault must 
then be isolated in order to take corrective action, 
such as disabling the faulting module. 

Error detection generally involves overhead in 
the form of additional processing time or logic. To 
minimize such overhead, a system should check 
for errors as infrequently as possible consistent 
with fault tolerant operation. At the very least, error 
checking must occur before data is outputted from 
CPU modules 30 and 30 . Otherwise, internal pro- 
cessing errors may cause improper operation in 
external systems, like a nuclear reactor, which is 
the condition that fault tolerant systems are de- 
signed to prevent. 

There are reasons for additional error checking. 
For example, to isolate faults or errors It is desir- 
able to check the data received by CPU modules 
30 and 30 prior to storage or use. Otherwise, when 
erroneous stored data Is later accessed and addi- 
tional errors result. It becomes difficult or impos- 
sible to find the original source of errors, especially 
when the erroneous data has been stored for some 
time. The passage of time as well as subsequent 
processing of the erroneous data may destroy any 
trail back to the source of the error. 

"Error latency," which refers to the amount of 
time an error is stored prior to detection, may 
cause later problems as well. For example, a 
seldom-used routine may uncover a latent error 
when the computer system is already operating 
with diminished capacity due to a previous error. 
When the computer system has diminished capac- 
ity, the latent error may cause the system to crash. 

Furthermore, it is desirable in the dual rail 
systems of processing systems 20 and 20' to 
check for errors prior to transferring data to single 
rail systems, such as a shared resource like mem- 



ory. This is because there are no longer two in- 
dependent sources of data after such transfers, and 
if any error in the single rail system is later de- 
tected, then error tracing becomes difficult if not 

6 Impossible. The preferred method of error handling 
is set forth in an application filed this same date 
entitled, "Software Error Handling", having the at- 
torney Docket No. PD89-289/DEC-344 
(FINK/P8243EP). which is herein incorporated by 

70 reference. 



C. MODULE DESCRIPTION 

75 

1. CPU Module 

The elements of CPU module 30 which appear 
in Fig. 1 are shown in greater detail in Figs. 3 and 

20 4. Fig. 3 is a block diagram of the CPU module, 
and Fig. 4 shows block diagrams of CPU module 
30 and I/O module 100 as well as their intercon- 
nections. Only CPU module 30 will be described 
since the operation of and the elements included in 

25 CPU modules 30 and 30' are generally the same. 

CPU module 30 contains dual CPUs 10 and 50. 
CPUs 40 and 50 can be standard central process- 
ing units known to persons of ordinary skill. In the 
preferred embodiment, CPUs 40 and 50 are micro- 

30 processors manufactured by Digital Equipment 
Corporation, the assignee of this application. - 

Associated with CPUs 40 and 50 are cache 
memories 42 and 52. respectively, which are stan- 
dard cache RAMs of sufficient memory size for the 

35 ' CPUs. In the preferred embodiment, the cache 
RAM Is 4k X 64 bits. It is not necessary for the 
present Invention to have a cache RAM. however. 



40 2. Memory Module 

Preferably, CPU's 40 and 50 can share up to 
four memory modules 60. Fig. 5 is a block diagram 
of one memory module 60 shown connected to 

45 CPU module 30. 

During memory transfer cycles, status register 
transfer cycles, and EEPROM transfer cycles, each 
memory module 60 transfers data to and from 
primary memory controller 70 via a bidirectional 

50 data bus 85. Each memory module 60 also re- 
ceives address, control, timing, and ECC signals 
from memory controllers 70 and 75 via buses 80 
and 82. respectively. The address signals on buses 
80 and 82 include board, bank, and row and col- 

55 umn address signals that identify the memory 
board, bank, and row and column address involved 
in the data transfer. 

As shown In Fig. 5. each memory module 60 
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Includes a memory array 600, Each memory array 
600 is a standard RAM in which the DRAI\/Is are 
organized into eight banks of memory. In the pre- 
ferred embodiment, fast page mode type DRAMs 
are used. 

Memory module 60 also includes control logic 
610, data transceivers/registers 620, memory dri- 
vers 630.. and an EEPROM 640. Data 
transceivers/receivers 620 provide a data buffer 
and data interface for- transferring data between 
memory array 600 and the bidirectional data lines 
of data bus 85. Memory drivers 630 distribute row 
and column address signals and control signals 
from control logic 610 to each bank in memory 
array 600 to enable transfer of a longword of data 
and Its corresponding ECC signals to or from the 
memory bank selected by the memory board and 
bank address signals. 

EEPROM 640, which can be any type of 
NVRAM {nonvolatile RAM), stores memory error 
data for off-line repair and configuration data, such 
as module size. When the memory module is re- 
moved after a fault, stored data Is extracted from 
EEPROM 640 to determine the cause of the fault. 
EEPROM 640 is addressed via row address lines 
from drivers 630 and by EEPROM control signals 
from control logic 610. EEPROM 640 transfers 
eight bits of data to and from a thirty-two bit 
internal memory data bus 645. 

Control logic 610 routes address signals to the 
elements of memory module 60 and generates 
Internal timing and control signals. As shown in 
greater detail in Fig. 6, control logic 610 includes a 
primary/mirror designator circuit 612. 

Primary/mirror designator circuit 612 receives 
two sets of memory board address, bank address, 
row and column address, cycle type, and cycle 
timing signals from memory controllers 70 and 75 
on buses 80 and 82, and also transfers two sets of 
ECC signals to or from the memory controllers on 
buses 80 and 82. Transceivers/registers in desig- 
nator 612 provide a buffer and interface for trans- 
ferring these signals to and from memory buses 80 • 
and 82. A primary/mirror multiplexer bit stored in 
status registers 618 indicates which one of memory 
controllers 70 and 75 is designated as the primary 
memory controller and which Is designated as the 
mirror memory controller, and a primary/mirror 
multiplexer signal is provided from status registers 
618 to designator 612. 

Primary/mirror designator 612 provides two 
sets of signals for distribution in control logic 610. 
One set of signals includes designated primary 
memory board address, bank address, row and 
column address, cycle type, cycle timing, and ECC 
signals. The other set of signals includes des- 
ignated mirror memory board address, bank ad- 
dress, row and column address, cycle type, cycle 
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timing, and ECC signals. The primary/mirror mul- 
tiplexer signal is used by designator 612 to select 
whether the signals on buses 80 and 82 will be 
respectively routed to the lines for carrying des- 

5 ignated primary signals and to the lines for carrying 
designated mirror signals, or vice-versa, 

A number of time division multiplexed bidirec- 
tional lines are included In buses 80 and 82. At 
certain times after the beginning of memory trans- 

70 fer cycles, status register transfer cycles, and 
EEPROM transfer cycles, ECC signals correspond- 
ing* to data on data bus 85 are placed on these ^ 
time division multiplexed bidirectional lines. If the 
transfer cycle is a write cycle, memory module 60 

15 receives data and ECC signals from the memory 
controllers. If the transfer cycle Is a read cycle, 
memory module 60 transmits data and ECC sig- 
nals to the memory controllers. At other times 
during transfer cycles, address, control, and timing 

20 signals are received by memory module 60 on the 
time division multiplexed bidirectional lines. Prefer- 
ably, at the beginning of memory transfer cycles, 
status register transfer cycles, and EEPROM trans- 
. fer cycles, memory controllers 70 and 75 transmit 

25 memory board address, bank address, and cycle 
type signals on these timeshared lines to each 
memory module 60. 

Preferably, row address signals and column 
. address signals are multiplexed on the same row 

30 and column address lines during transfer cycles. 
First, a row address is provided to memory module 
60 by the memory controllers, followed by a col- 
umn address about sixty nanoseconds later. 

A sequencer 616 receives as inputs a system 

35 clock signal and a reset signal from CPU module 
30, and receives the designated primary cycle tim- 
. ing. designated primary cycle type, designated mir- 
ror cycle timing, and designated mirror cycle type 
signals from the transceivers/registers in designator 

40 612. 

Sequencer 616 is a ring counter with asso- 
ciated steering logic that generates and distributes 
a number of control and sequence timing signals 
for the memory module that are needed In order to 

45 execute the various types of cycles. The control 
and sequence timing signals are generated from 
the system clock signals, the designated primary 
cycle timing signals, and the designated primary 
cycle type signals. 

50 Sequencer 616 also generates a duplicate set 
of sequence timing signals from the system clock 
signals, the designated mirror cycle timing signals, 
and the designated mirror cycle type signals. 
These duplicate sequence timing signals are used 

55 for error checking. For data transfers of multi-long 
words of data to and from memory module 60 In a 
fast page mode, each set of column addresses 
starting with the first set is followed by the next 
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column address 120 nanoseconds later, and each 
long word of data is moved across bus 85 120 
nanoseconds after the previous long word of data. 

Sequencer 616 also generates tx/rx register 
control signals. The tx/rx register control signals 
are provided to control the operation of data- 
transceivers/registers 620 and the 
transceivers/registers in designator 612. The direc- 
tion of data flow is determined by the steering logic 
In sequencer 616, which responds to the des- 
ignated primary cycle type signals by generating 
tx/rx control and sequence timing signals to in- 
dicate whether and when data and ECC signals 
should be written into or read from the 
transceivers/registers in memory module 60. Thus, 
during memory write cycles, status register write 
cycles, and EEPROM write cycles, data and ECC 
signals will be latched into the 
transceivers/registers from buses 80, 82, and 85, 
while during memory read cycles, status register 
read cycles, and EEPROM read cycles, data and 
ECC signals will be latched into the 
transceivers/registers from memory array 600, sta- 
tus registers 618, or EEPROM 640 for output to 
CPU module 30. 

Sequencer 616 also generates EEPROM con- 
trol signals to control the operation of EEPROM 
640. 

The timing relationships that exist in memory 
module 60 are specified with reference to the rise 
time of the system clock signal, which has a period 
of thirty nanoseconds. All status register read and 
write cycles, and all memory read and write cycles 
of a single longword, are performed in ten system 
clock periods, i.e., 300 nanoseconds. Memory read 
and write transfer cycles may consist of multl- 
longword transfers. For each additional longword 
that is transferred, the memory transfer cycle is 
extended for four additional system clock periods. 
Memory refresh cycles and EEPROM write cycles 
require at least twelve system clock periods to 
execute, and EEPROM read cycles require at least 
twenty system clock periods. 

The designated primary cycle timing signal 
causes sequencer 616 to start generating the se- 
quence timing and control signals that enable the 
memory module selected by the memory board 
address signals to implement a requested cycle. 
The transition of the designated primary cycle tim- 
ing signal to an active state marks the start of the 
cycle. The return of the designated primary cycle 
timing signal to an inactive state mari(s the end of 
the cycle. 

The sequence timing signals generated by se- 
quencer 616 are associated with the different states 
entered by the sequencer as a cycle requested by 
CPU module 30 is executed. In order to specify the 
timing relationship among these different states 



(and the timing relationship among sequence tim- 
ing signals corresponding to each of these states), 
the discrete states that may be entered by se- 
quencer 616 are Identified as states SEQ' IDLE and 

5 SEQ 1 to* SEQ 19. Each state lasts for a single 
system clock period (thirty nanoseconds). Entry by 
sequencer 616 into each different state is triggered 
by the leading edge of the system clock signal. 
The leading edges of the system clock signal that 

10 cause sequencer 616 to enter states SEQ IDLE 
and SEQ 1 to SEQ 1.9 are referred to as transitions 
! T IDLE and T1 to T19 to relate them to the se- 
quencer states, i.e.. TN is the system clock signal 
leading edge that causes sequencer 616 to enter 

15 state SEQ N. 

At times when CPU module 30 Is not directing 
memory module 60 to execute a cycle, the des- 
ignated primary cycle timing signal is not asserted, 
and the sequencer remains in state SEQ IDLE. The 

20 sequencqr is started (enters state SEQ 1) in re- 
sponse to assertion by memory controller 70 of the 
cycle timing signal on bus 80, provided control 
logic 610 and sequencer 616 are located in the 
memory module selected by memory board ad- 

25 dress signals also transmitted from memory con- 
troller 70 on bus 80. The rising edge of the first 
system clock signal following assertion of the des- 
ignated primary cycle active signal corresponds to 
transition T1. 

30 As indicated previously, in the case of transfers 
of a single longword to or from memory array 600. 
the cycle is performed in ten system clock periods. 
The sequencer proceeds from SEQ IDLE, to states 
SEQ 1 through SEQ 9, and returns to SEQ IDLE, 

35 Memory read and write cycles may be ex- 
tended, however, to transfer additional longwords. 
Memory array 600 preferably uses "fast page 
mode" DRAMs. During multi-longword reads and 
writes, transfers of data to and from the memory 

40 array after transfer of the first longword are accom- 
plished by repeatedly updating the column address 
and regenerating a CAS (column address strobe) 
signal. 

During multi-longword transfer cycles, these 
45 updates of the column address can be implement- 
ed because sequencer 616 repeatedly loops from . 
states SEQ 4 through SEQ 7 until all of the long- 
words are transferred. For example, if three long- 
words are being read from or written into memory 
50 array 600, the sequencer enters states SEQ IDLE, 
SEQ 1, SEQ 2. SEQ 3. SEQ 4. SEQ 5. SEQ 6, 
SEQ 7. SEQ 4, SEQ 5, SEQ 6, SEQ 7. SEQ 4, 
SEQ 5. SEQ 6. SEQ 7, SEQ 8. SEQ 9, and SEQ 
IDLE. 

55 During a memory transfer cycle, the desig- 

nated primary cycle timing signal is monitored by 
sequencer 616 during transition T6 to determine 
whether to extend the memory read or write cycle 
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in order to transfer at least one additional tongword. 
At times when the designated primary cycle timing 
signal is asserted during transition T6, the se- 
quencer in state SEQ 7 will respond to the next 
system clock signal by entering state SEQ 4 in* 
stead of entering state SEQ 8. 

In the case of a multi-longword transfer, the 
designated primary cycle timing signal is asserted 
at least fifteen nanoseconds before the first T6 
transition and remains asserted until the final long- 
word is transferred. In order to end a memory 
transfer cycle after the final long word has been 
transferred, the designated primary cycle timing 
signal is deasserted at least fifteen nanoseconds 
before the last. T6 transition and remains deasser- 
ted for at least ten nanoseconds after the last T6 
transition. 

During memory transfer cycles, the designated 
primary row address signals and the designated 
primary column address signals are presented at 
different times by designator 612 in control logic 
610 to memory drivers 630 on a set of time di- 
vision multiplexed lines. The outputs of drivers 630 
are. applied to the address inputs of the DRAMs in 
memory array 600, and also are returned to control 
logic 610 for comparison with the designated min^or 
row and column address signals to check for er- 
rors. During status register transfer cycles and 
EERROM transfer cycles, column address signals 
are not needed to select a particular storage loca- 
tion. 

During a memory transfer cycle, row address 
signals are the first signals presented on the 
timeshared row and column address lines of buses 
80 and 82. During state SEQ IDLE, row address 
signals are transmitted by the memory controllers 
on the row and column address lines, and the row 
address is stable from at least fifteen nanoseconds 
before the T1 transition until ten nanoseconds after 
the T1 transition. Next, column address signals are 
transmitted by the memory controllers on the row 
and column address lines, and the column address 
is stable from at least ten nanoseconds before the 
T3 transition until fifteen nanoseconds after the T4 
transition. In the case of multi-longword transfers 
during memory transfer cycles, subsequent column 
address signals are then transmitted on the row 
and column address lines, and these subsequent 
column addresses are stable from ten 
nanoseconds before the T6 transition until fifteen 
nanoseconds after the T7 transition. 

Generator/checker 817 receives the two sets of 
sequence timing signals generated by sequencer 
616. In addition, the designated primary cycle type 
and bank address signals and the designated mir- 
ror cycle type and bank address signals are trans- 
mitted to generator/checker 617 by designator 612. 
In the generator/checker, a number of primary con- 



trol signals, i.e., RAS (row address strobe). CAS 
(column address strobe), and WE (write enable), 
are generated for distribution to drivers 630. using 
the primary sequence timing signals and the des- 

5 ignated primary cycle type and bank address sig- 
nals. A duplicate set of these control signals is 
generated by generator/checker 617 from the du- 
plicate (mirror) sequence timing signals and the 
designated mirror cycle type and bank address 

70 signals. These mirror RAS, CAS, and write enable 
signals are used for error checking. 

When the primary cycle type signals indicate a 
memory transfer cycle is being performed, the 
primary bank address signals identify one selected 

75 bank of DRAMs in memory array 600. Memory 
drivers 630 include separate RAS drivers for each 
bank of DRAMs in memory array 600. In 
generator/checker 617, the primary RAS signal is 
generated during the memory transfer cycle and 

20 demultiplexed onto one of the lines connecting the 
generator/checker to the RAS drivers. As a result, 
only the RAS driver corresponding to the selected 
DRAM bank receives an asserted RAS signal dur- 
ing the memory transfer cycle. During refresh cy- 

25 cles, the primary RAS signal is not demultiplexed 
and an asserted RAS signal Is received by each 
RAS driver. During status register transfer cycles 
and EEPROM transfer cycles, the bank address 
signals are unnecessary. 

30 Memory drivers 630 also include CAS drivers. 

In generator/checker 617, the primary CAS signal 
is generated during memory transfer cycles and 
refresh cycles. The primary CAS signal is not de- 
multiplexed and an asserted CAS signal is received 

35 by each CAS driver. 

During memory write cycles, the primary WE 
signal is generated by generator/checker 617. The 
asserted WE signal is provided by drivers 630 to 
each DRAM bank in memory array 600. However, a 

40 write can only be executed by the selected DRAM 
bank, which also receives asserted RAS and CAS 
signals. 

In the preferred embodiment of the invention, 
during memory transfer cycles the primary RAS 

45 signal Is asserted during the T2 transition, is stable 
from at least ten nanoseconds before the T3 transi- 
tion, and is deasserted during the last T7 transition. 
The primary CAS signal is asserted fifteen 
nanoseconds after each T4 transition, and is deas- 

60 serted during each T7 transition. During memory 
write cycles the primary WE signal is asserted 
during the T3 transition, is stable from at least ten 
nanoseconds before the first T4 transition, and Is 
deasserted during the last T7 transition. 

55 When the primary cycle type signals indicate 
memory refresh cycle is being performed, 
generator/checker 617 causes memory array 600 
to perform memory refresh operations in response 
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to the primary sequence timing signals provided by 
sequencer 616. During these refresh operations, 
the RAS and CAS signals are generated and dis- 
tributed by the generator/checl<er in reverse order. 
This mode of refresh requires no extemal address- 
ing for bank, row, or column. 

During transfer cycles, EGG signals are trans- ' 
ferred on the time division multiplexed bidirectional 
lines of buses 80 and 82 at times when data is 
being transferred on bus 85. However, these same 
lines are used to transfer control (e.g.. cycle type) 
and address (e.g.. memory board address and 
bank address) signals at other times during the 
transfer cycle. 

The transceivers/registers in primary/mirror de- 
signator 612 include receivers and transmitters that 
are responsive to sequence timing signals and tx/rx 
register control signals provided by sequencer 616. 
The sequence timing signals and tx/rx register con- 
trol signals enable multiplexing of EGG signals and 
address and control signals on the time division 
multiplexed bidirectional lines of buses 80 and 82. 

Preferably, control and address signals, such 
as cycle type, memory board address, and bank 
address signals, are transmitted by memory con- 
trollers 70 and 75 and presented on the timeshared 
lines of buses 80 and 82 at the beginning of either 
single or multi-Iongword transfer cycles. These sig- 
nals start their transition (while the sequencer is in 
the SEQ IDLE state) concurrent with activation of 
the cycle timing signal, and remain stable through 
T2. Therefore, in the transceivers/registers of de- 
signator 612, the receivers are enabled and the 
transmitters are set Into their tristate mode at least 
until the end of state SEQ 2. 

The cycle type signals identify which of the 
following listed functions will be performed by 
memory array 60 during the cycle: memory read, 
memory write, status register read, status register 
write, EEPROM read, EEPROM write, and refresh. 
The designated prinriary cycle type signals re- 
ceived by designator 612 are provided to sequenc- 
er 616 and used in generating tx/rx control signals 
and sequence timing signals. For example, in data 
transceivers/regiisters 620 and In the 
transceivers/registers of designator 612, the receiv- 
ers are enabled and the transmitters are set into 
their tristate mode by sequencer 616 throughout a 
write cycle. However, in data transceivers/registers 
620 and In tiie transceivers/registers of designator 
612 during a read cycle, tiie receivers are set into 
their tristate mode and the transmitters are enabled 
by sequencer 616 after the cycle type, memory 
board address, and bank address signals have 
been received at the beginning of the cycle. 

In the prefenred embodiment, data transfen-ed 
to or from memory anray 600 is checked in each 
memory module 60 using an Error Detecting Code 



(EDG). which is preferably the same code required 
by memory controllers 70 and 75. The preferred 
code Is a single bit correcting, double bit detecting, 
error correcting code (EGG). 

5 During a memory write cycle, memory control- 

ler 70 transmits at least one longword of data on 
data bus 85 and simultaneously transmits a cor- 
responding set of EGG signals on bus 80. Mean- 
white, memory controller 75 transmits a second set 

70 of EGG signals, which also correspond to the long- 
word on data bus 85, on bus 82. 

As embodied herein, during a memory write 
cycle tiie data and the EGG signals for each long- 
word are presented to the receivers of data 

75 transceivers/registers 620 and to the receivers of 
the transceivers/registers of designator 612. The 
data and the EGG signals, which are stable at least 
ten nanoseconds before the T4 transition and re- 
main stable until fifteen nanoseconds after the T6 

20 transition, are latched into these 
transceivers/registers. During this time period, 
memory controllers 70 and 75 do not provide ad- 
dress and control signals on the timeshared lines 
of buses 80 and 82. 

25 The designated primary EGG signals received 
by designator 612 and the longword of data re- 
ceived by transceivers/registers 620 during the 
memory write cycle are provided to the data inputs 
of the DRAMs in each of the eight banks of mem- 

30 ory array 600 and to EGG generator 623. The 
generated EGG Is compared to the designated 
primary EGG by comparator 625. The designated 
primary EGG signals also are provided to EGG 
comparators 625, together with the designated mir- 

36 Tor EGG signals. 

As embodied herein, during a memory read 
cycle, at least one longword of data and a cor- 
responding set of EGG signals are read from mem- 
ory array 600 and respectively steered to data 

40 transceivers/registers 620 and to the 
transceivers/registers of designator 612. During 
transition T7 of the memory read cycle, the data 
and the EGG signals for each longword are avail- 
able from memory array 600 and are latched into 

45 these transceivers/registers. The data is also pre- 
sented to the EGG generator 623 and its output is 
compared to the EGG read from memory. 

After latching, the data and tiie EGG signals 
are presented to data bus 85 and to buses 80 and 

60 82 by the transmitters of data transceivers/registers 
620 and by the transmitters of tiie 
transceivers/registers of designator 612. The same 
EGG signals are transmitted from tiie 
transceivers/registers in designator 612 to memory 

55 controller 70 and to memory controller 75. The 
data and the EGG signals transmitted on data bus 
85 and on buses 80 and 82 are stable from fifteen 
nanoseconds after the T7 transition until five 
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nanoseconds before the following T6 transition (In 
the case of a multi-Iongword transfer) or until five 
nanoseconds before the following T IDLE transition 
(in the case of a single longword. transfer or the last 
longword of a multi-Iongword transfer). During this 
time period, memory controllers 70 and .75 do not 
provide address and control signals on the 
timeshared lines of buses 80 and 82. The transmit- 
ters of data transceivers/registers 620 and the 
transmitters of the transceivers/registers of desig- 
nator 612 are set into their tristate mode during the 
following T IDLE transition. 

Comparator 614 Is provided to compare the 
address, control, and timing signals originating 
from controller 70 with the corresponding address, 
control, and timing signals originating from control- 
ler 75. The designated primary cycle timing sig- 
nals, cycle type signals, memory board' address 
signals, and bank address signals, together with 
the designated mirror cycle timing signals, cycle 
type signals, memory board address signals. banl< 
address signals, row address signals, and column 
address signals, are provided from designator 612 
to comparator 614. The designated primary row 
address signals and column address signals are 
provided from the outputs of drivers 630 to com- 
parator 614. Both sets of signals are then com- 
pared. 

If there is a miscompare between any of the 
address, control, and timing signals originating 
from the memory controllers, comparator 614 gen- 
erates an appropriate error signal. As shown in 
Figure 6, board address error, bank address en-or. 
row address error, column address error, cycle 
type address error and cycle timing error signals 
may be output by the comparator. 

Generator/checker 617 compares the primary 
control and timing signals generated by sequencer 
616 and generator/checker 617 using the desig- 
nated primary bank address, cycle type, and cycle 
timing signals with the mln^or control and timing 
signals generated using the designated minror bank 
address, cycle type, and cycle timing signals. The 
two sets of sequence timing signals are provided 
by sequencer 616 to generator/checker 617. The 
primary RAS. CAS, and WE signals are provided 
from the outputs of drivers 630 to generator/ check- 
er 617. As indicated previously, the mirror RAS, 
CAS. and WE signals are generated intematly by 
the generator/checker.. Generator/checker 617 com- 
pares the primary RAS. CAS, WE, and sequence 
timing signals to the mirror RAS. CAS, WE. and 
sequence timing signals. 

If there is a miscompare between any of the 
control and timing signals originating from se- 
quencer 616 or generator/checker 617, the 
generator/checker generates an appropriate error 
signal. As shown In Figure 6. sequencer enror, RAS 
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error, CAS en'or, and WE enror signals may be 
output by generator/checker 617. 

Error signals are provided from comparator 614 
and from generator/checker 617 to address/control 

5 error logic 621. In response to receipt of an error 
signal from comparator 614 or from 
generator/checker 617, address/control enror logic 
621 transmits an address/control enror signal to 
CPU module 30 to indicate the detection of a fault 

TO due to a miscompare between any address, con- 
trol, or timing signals. The address/control error 
signal is sent to error logic in memory controllers 
70 and 75 for error handling. The transmission of 
the address/control en'or signal to CPU module 30 

75 causes a GPU/MEM fault, which is discussed in 
greater detail in other sections. 

The error signals from comparator 614 and 
from generator/checker 617 also are provided to 
status registers 618. In the status registers, the 

20 error signals and all of the address, control, timing, 
data, and ECC signals relevant to the fault are 
temporarily stored to enable en-or diagnosis and 
recovery. 

In accordance with one aspect of the invention. 

25 only a single thirty-two bit data bus 85 is provided 
between CPU module 30 and memory module 60. 
Therefore, memory module 60 cannot compare two 
sets of data from memory controllers 70 and 75. 
I However, data integrity is verified by memory mod- 

30 uie 60 without using a duplicate set of thirty-two 
data lines by checking the two separate sets of 
ECC signals that are transmitted by memory con- 
trollers 70 and 75 to memory module 60. 

As shown in Rg. 6, control logic 610 includes 

35 ECC generator 623 and ECC comparators 625. The 
designated primary and mirror ECC signals are 
provided by designator 612 to the ECC compara- 
tors. During a memory write cycle, the designated 
primary ECC signals are compared to the des- 

40 ignated mirror ECC signals. As a result, memory 
module 60 verifies whether memory controllers 70 
and 75 are in agreement and whether the des- 
' Ignated primary ECC signals being stored In the 
DRAMs of memory array 600 during the memory 

4S write cycle are con-ect. Furthermore, the data pre- 
sented to the data inputs of the DRAMs during the 
memory write cycle is provided to ECC generator 
623. ECC generator 623 produces a set of gen- 
erated ECC signals that correspond to the data and 

50 provides the generated ECC signals to ECC com- 
parators 625. The designated primary ECC signals 
are compared to the generated ECC signals to 
verify whether the data transmitted on data bus 85 
by memory controller 70 is the same as the data 

55 being stored in the DRAMs of memory array 600. 

During a memory read cycle, the data read 
from the selected b^nk of DRAMs is presented to 
the ECC generator. The generated ECC signals 
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then are provided to the ECC comparators, which 
also receive stored ECC signals read from the. 
selected bank of DRAMs. The generated and 
stored ECC signals are compared by ECC com- 
parators 625. 

If there is a miscompare between any of pairs 
of ECC signals monitored by ECC comparators 
625, the ECC comparators generate an appropriate 
error signal. As shown In Figure 6, primary/mirror 
ECC error, primary/generated ECC error, and 
memory/generated ECC error signals may be out- 
put by the ECC comparators. 

These ECC enror signals from ECC compara- 
tors 625 are provided to status registers 618. In the 
status registers, each of the ECC error signals and 
all of the address, control, timing, data, and ECC 
signals relevant to an ECC fault are temporarily 
stored to enable error diagnosis and recovery. 

An ECC error signal is asserted by ECC com- 
parators 625 on an ECC error line and transmitted 
to CPU module 30 to Indicate the detection of an 
ECC fault due to a miscompare. The miscompare 
can occur during either of the two ECC checks 
performed during a memory write cycle, or during 
the single ECC check performed during a memory 
read cycle. 

As shown in Figure 6, board select logic 627 
receives slot signals from a memory backplane. 
The slot signals specify a unique slot location for 
each memory module 60. Board select logic 627 
then compares the slot signals with the designated 
primary board address signals transmitted from 
one of the memory controllers via designator circuit 
612. A board selected signal is generated by board 
select logic 627 if the slot signals are the same as 
the designated primary board address signals, 
thereby enabling the other circuitry in control logic 
610. 



3. Memory Controller 

Memory controllers 70 and 75 control the ac- 
cess of CPUs 40 and 50, respectively, to memory 
module 60, auxiliary memory elements and, in the 
preferred embodiment, perform certain error han- 
dling operations. The auxiliary memory elements 
coupled to memory controller 70 include system 
ROM 43. EEPROM 44. and scratch pad RAM 45. 
ROM 43 holds certain standard code, such as 
diagnostics, console drivers, and part of the boot- 
strap code. EEPROM 44 is used to hold informa- 
tion such as error information detected during the 
operation of CPU 40, which may need to be modi- 
fied, but which should not be lost when power is 
removed. Scratch pad RAM 45 is used for certain 
operations performed by CPU 40 and to convert 
rail-unique information (e.g.. information specific to 



conditions on one rail which is available to only one 
CPU 40 or 50) to zone information (e.g., informa- 
tion which can be accessed by both CPUs 40 and 
50), 

5 Equivalent elements 53. 54 and 55 are coupled 

to memory controller 75. System ROM 53, EEPR- 
OM 54, and scratch pad RAM 55 are the same as 
system ROM 43, EEPROM 44, and scratch pad 
RAM 45, respectively, and perform the same func- 

70 tlons. 

The details of the preferred embodiment of 
primary memory controller 70 can be seen in Figs. 
7-9. Mirror memory controller 75 has the same 
elements as shown in Rgs. 7-9, but differs slightly 

76 in operation. Therefore, only primary memory con- 
troller 70's operation will be described, except 
where the operation of memory controller 75 dif- 
fers. Memory controllers 70 and 75 in processing 
system 20 have the same elements and act the 

20 same as memory controllers 70 and 75, respec- 
tively. 

The elements shown in Fig. 7 control the flow 
Of data, addresses and signals through primary 
memory controller 70. Control logic 700 controls 

25 the state of the various elements in Fig. 7 accord- 
ing to the signals received by memory controller 
70 and the state engine of that memory controller 
which is stored in control logic 700. Multiplexer 702 
selects addresses from one of three sources. The 

30 addresses can either come from CPU 30 via re- 
ceiver 705. from the DMA engine 800 described 
below in reference to Fig. 8, or from a refresh 
resync address line which is used to generate an 
artificial refresh during certain bulk memory trans- 

35 fers from one zone to. another during resynch- 
ronization operations. 

The output of multiplexer 702 is an input to 
multiplexer 710, as is data from CPU 30 received 
via receiver 705 and data from DMA engine 800. 

40 The output of multiplexer 710 provides data to 
memory module 60 via memory interconnect 85 
and driver 715. Driver 715. is disabled for mirror 
memory control modules 75 and 75' because only 
one set of memory data is sent to memory mod- 

45 ules 60 and 60 . respectively. 

The data sent to memory interconnect 85 In- 
cludes either data to be stored in memory module 
60 from CPU 30 or DMA engine 800. Data from 
CPU 30 and addresses from multiplexer 702 are 

50 also sent to DMA engine 800 via this path and also 
via receiver 745 and ECC corrector 750. 

The addresses from multiplexer 702 also pro- 
vide an input to demultiplexer 720 which divides 
the addresses into a row/column address portion, a 

55 board/bank address portion, and a single board bit. 
The twenty-two bits of the row/column address are 
multiplexed onto eleven lines. In the preferred em- 
bodiment, the twenty-two row/column address bits 
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are sent to memory module 60 via drivers 721. The 
single board bit is preferably sent to memory mod- 
ule 60 via driver 722. and the other board/bank 
address bits are multiplexed with ECC signals. 

Multiplexer 725 combines a normal refresh 
command for memory controller 70 along with cy- 
cle type information from CPU 30 (i.e., read, write, 
etc.) and DMA cycle type Information. The normal 
refresh command and the refresh resync address 
both cause memory module 60 to initiate a mem- 
ory refresh operation. 

The output of multiplexer 725 is an input to 
multiplexer 730 along with the board/bank address 
from demultiplexer 720. Another input into mul- 
tiplexer 730 is the output of ECC generator/checker 
735. Multiplexer 730 selects one of the inputs and 
places it on the time-division multiplexed 
ECC/address lines to memory module 60. Mul- 
tiplexer 730 allows those time-division multiplexed 
lines to carry board/bank address and additional 
control information as well as ECC infonmation. 
although at different times. 

ECC information Is received from memory 
modules 60 via receiver 734 and is provided as an 
input to ECC generator/checker 735 to compare 
the ECC generated by memory module 60 with 
that generated by memory controller 70. 

Another input into ECC generator/checker 735 
is the output of multiplexer 740. Depending upon 
whether the memory transaction is a write transac- 
tion or a read transaction, multiplexer 740 receives 
as inputs the memory data sent to memory module 
60 from multiplexer 710 or the memory data re- 
ceived from memory module 60 via receiver 745. 
Multiplexer 740 selects one of these sets of mem- 
ory data to be the Input to ECC generator/checker 
735. Generator/checker 735 then generates the ap- 
propriate ECC code which, in addition to being sent 
to multiplexer 730, is also sent to ECC corrector 
750. In the preferred embodiment, ECC corrector 
750 corrects any single bit en-ors in the mismory 
data received from memory module 60. 

The corrected memory data from ECC checker 
750 is then sent to the DMA engine shown in Fig. 8 
as well as to multiplexer 752. The other Input into 
multiplexer 752 is error information from the error 
handling logic described below in connection with 
Rg. 9. The output of multiplexer 752 Is sent to 
CPU 30 via driver 753. 

Comparator 755 compares the data sent from 
multiplexer 710 to memory module 60 with a copy 
of that data after it passes through driver 715 and 
receiver 745. This checking determines whether 
driver 715 and receiver 745 are operating correctly. 
The output of comparator 755 is a CMP error 
signal which indicates the presence or absence of 
such a comparison error. The CMP error feeds the 
error logic in Fig. 9. 



Two other elements in Fig. 7 provide a different 
kind of enror detection. Element 760 is a parity 
generator. ECC data, generated either by the mem- 
ory controller 70 on data to be stored in memory 

5 nnpdule 60 or generated by memory module 60 on 
data read from memory module 60 is sent to a 
parity generator 760. The parity signal from gener- 
ator 760 Is sent, via driver 762, to comparator 765. 
Comparator 765 compares the ECC parity signal 

10 from generator 760 with an equivalent ECC parity 
signal generated by controller 75'. 

Parity generator 770 performs the same type 
of a check on the row/column and single bit board 
address signals received from demultiplexer 720. 

76 The address parity signal from parity generator 770 
is transmitted by a driver 772 to a comparator 775 
which also receives an address parity signal from 
controller 75. The outputs of comparator 765 and 
775 are parity error signals which feed the en-or 

20 logic in Fig. 9. 

Rg. 8 shows the fundamentals of a DMA en- 
gine 800. In the preferred embodiment, DMA en- 
gine 800 resides In memory controller 70. but there 
is no requirement for such placement. As shown in 

26 Fig. 8, DMA engine 800 Includes a data router 810, 
a DMA control 820, and DMA registers 830. Driver 
815 and receiver 816 provide an Interface between 
memory controller 70 and cross-link 90. 

DMA control 820 receives Iritemal control slg- 

30 nals from control logic 700 and, In response, sends 
control signals to place data router 810 Into the 
appropriate configuration. Control 820 also causes 
data router 810 to set Its configuration to route data 
and control signals from cross-link 90 to the mem- 

35 ory control 70 circuitry shown in Fig. 7. Data router 
810 sends Its status signals to DMA control 820 
which relays such signals, along with other DMA 
information, to error logic in Rg. 9. 

Registers 830 Includes a DMA byte counter 

40 register 832 and a DMA address register 836. 
These registers are set to initial values by CPU 40 
via router 810. Then, during DMA cycles, control 
820 causes, via router 810. the counter register 832 
to increment and address register 836 to decre- 

45 ment. Control 820 also causes the contents of 
address registers 836 to be sent to memory mod- 
ule 60 through router 810 and the circuitry In Rg. 7 
during DMA operations. 

As explained above, in the preferred embodi- 

60 ment of this invention, the memory controllers 70, 
• 75, 70 and 75 also perform certain fundamental 
error operations. An example of the preferred em- 
bodiment of the hardware to perform such error 
operations are shown in Rg. 9. 

55 As shown in Rg. 9, certain memory controller 
internal signals, such as timeout. ECC error and 
bus miscompare, are inputs into diagnostic error 
logic 870, as are certain external signals such as 
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rail error, firewaii miscompare, and address/control 
error. In the prefen'ed embodiment, diagnostic enror 
logic 870 receives error signals from the other 
components of system 10 via cross-links 90 and 
95. 

Diagnostic error logic 870 forms error pulses 
from the error signals and from a control pulse 
signal generated from the basic timing of memory 
controller 70. The error pulses generated by di- 
agnostic error logic 870 contain certain error in- 
formation which is stored into appropriate locations 
in a diagnostic error register 880 in accordance 
with certain timing signals. System fault error ad- 
dress register 865 stores the address in memory, 
module 60 which CPUs 40 and 50 were commu- 
nicating with when an error occurred. 

The error pulses from diagnostic error logic 
870 are also sent to error categorization logic 850 
which also receives information from CPU 30 in- 
dicating the cycle type (e.g., read, write, etc.). 
From that information and the error pulses, error 
categorization logic 850 determines the presence 
of CPU/I/O errors, DMA errors, or CPU/MEM faults. 

A CPU/I/O error is an error on an operation that 
is directly attributable to a CPU/I/O cycle on bus 46 
and may be hardware recoverable, as explained 
below in regard to resets. DMA errors are errors 
that occur during a DIVIA cycle and, in the pre- 
ferred embodiment, are handled principally by soft- 
ware. CPU/MEM faults are errors that for which the 
correct operation of CPU or the contents of mem- 
ory cannot be guaranteed. 

The outputs from en^or categorization logic 850 
are sent to encoder 855 which forms a specific, 
error code. This error code is then sent to cross- 
links 90 and 95 via AND gate 856 when the error 
disable signal is not present. 

After receiving the error codes, cross-links 90, 
95, 90 and 95 send a retry request signal back to 
the meniory controllers. As shown in Fig. 9, an 
encoder 895 in memory controller 70 receives the 
retry request signal along with cycle type informa- 
tion and the error signals (collectively shown as 
cycle qualifiers). Encoder 895 then generates an 
appropriate en'or code for storage In a system fault 
error register 898. 

System fault error register 898 does not store 
the same information as diagnostic error register 
880. Unlike the system fault error register 898. the 
diagnostic error register 880 only contains rail 
unique information, such as an error on one input 
from a cross-link rail, and zone unique data, such 
as an uncorrectable ECC enror in memory module 
60. 

System fault error register 898 also contains 
several bits which are used for error handling. 
These include a NXM bit indicating that a desired 
memory location is missing, a NXIO bit indicating 
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that a desired 1/0 location is missing, a solid fault 
bit and a transient bit. The transient and solid bits 
* together indicate the fault level. The transient bit 
also causes system fault error address register 865 

5 to freeze. 

Memory controller status register 875. although 
technically not part of the error logic, is shown in 
Fig. 9 also. Register 875 stores certain status in- 
formation such as a DMA ratio code in DMA ratio 

10 portion 877, an error disable code In error disable 
portion 878, and a mirror bus driver enable code in 
mirror bus driver enable portion 876. The DMA 
ratio code specifies the fraction of memory band- 
width which can be allotted to DMA, The error 

75 disable code provides a signal for disabling AND 
gate 856 and thus the error code. The mirror bus 
driver enable code provides a signal for enabling 
the mirror bus drivers for certain data transactions. 

20 

4. Cross-link 

Data for memory resync. DMA and I/O oper- 
ations pass through cross-links 90 and 95. Gen- 

25 erally. cross-links 90 and 95 provide communica- 
tions between CPU module 30, CPU module 30', 
1/0 modules 100, 110, 120. and 1/0 modules lOO', 
110'. 120' (see Fig. 1). 

Cross-links 90 and 95 contain both parallel 

30 registers 910 and serial registers 920 as shown in 
Fig. 10, Both types of registers are used for inter- 
processor communication in the preferred embodi- 
ment of this invention. During normal operation, 
processing systems 20 and 20' are synchronized 

35 and data is exchanged in parallel between process- 
ing systems 20 and 20' using parallel registers 910 
in cross-links 90/95 and 90'/95', respectively. When 
processing systems 20 and 20' are not synchro- 
nized, most notably during bootstrapping, data is 

40 exchanged between cross-links by way of serial 
registers 920. 

The addresses of the parallel registers are in 
I/O space as opposed to memory space. Memory 
space refers .to locations in memory module 60. 1/0 

45 space refers to locations such as 1/0 and internal 
system registers, which are not in memory module 
60. 

Within 1/0 space, addresses can either be in 
system address space or zone address space. The 

50 tenm "system address space" refers to addresses 
that are accessible throughout the entire system 
10. and thus by both processing systems 20 and 
20'. The term "zone address space" refers to 
addresses which are accessible only by the zone 

55 containing the particular cross-link. 

The parallel registers shown in Fig. 10 include 
a communications register 906 and an 1/0 reset 
register 908. Communications register 906 contains 
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unique data to be exchanged between zones. Such 
data Is usually zone-unique, such as a memory soft 
^ error (it is almost beyond the realm of probability 
that memory modules 60 and 60' would Indepen- 
dently experience the same error at the same 
time). 

Because the data to be stored Into register 906 
is unique, the address of communications register 
906 for purposes of writing must be In zone ad- 
dress space. Otherwise, processing systems 20 
and 20 . because they are in lockstep synchroniza- 
tion and executing the same series of Instruction at 
substantially the same time, could not store zone 
unique data into only the communications registers 
906 in zone 1 1 ; they would have to store that same 
data Into the communications registers 906' (not 
shown) in zone 11*. 

The address of communications register 906 
for reading, however, is in system address space. 
Thus, during synchronous operation, both zones 
can simultaneously read the communications regis- 
ter from one zone and then simultaneously read 
the communications register from the other zone, 

I/O reset register 908 resides in system ad- 
dress space. The I/O reset register includes one bit 
per I/O module to indicate whether the correspond- 
ing module is in a reset state. When an I/O module 
is in a reset state, it is effectively disabled. 

Parallel registers 910 also Include other regis- 
ters, but an understanding of those other registers 
is not necessary to an understanding of the present 
invention. 

All of the serial cross-link registers 920 are In 
the zone specific space since they are used either 
for asynchronous communication or contain only 
zone specific information. The purpose of the serial 
cross-link registers and the serial cross-link is to 
allow processors 20 and 20' to communicate even 
thougii they are not running in lockstep synchro- 
nization (i.e., phase-locked clocks and same mem- 
ory states). In the preferred embodiment, there are 
several serial registers, but they need not be de- 
scribed to understand this invention. 

Control and status register 912 is a serial regis- 
ter which contains status and control flags. One of 
the fiags is an OSR bit 913 which is used for 
bootstrapping and indicates whether the processing 
system In the corresponding zone has already be- 
gun its bootstrapping process or whether the op- 
erating system for that zone is currently running, 
either because Its bootstrapping process has com- 
pleted, or because it underwent a resynchroniza- 
tion. 

Control and status register 912 also contain the 
mode bits 914 for Identifying the current mode of 
cross-link 90 and thus of processing system 20. 
Preferably mode bits Include resync mode bits 915 
and cross-link mode bits 916. Resync mode bits 



915 identify cross-link 90 as being either in resync 
slave or resync master mode. The cross-link mode 
bits 916 identify cross-link 90 as being either in 
cross-link off, duplex, cross-link master, or cross- 

5 link slave mode. 

One of the uses for the serial registers is a 
status read operation which allows the cross-link In 
one zone to read the status of the other zone's 
cross-link. Setting a status read request flag 918 In 

10 serial control and status register 912 sends a re- 
quest for status information to cross-link 90'. Upon 
receipt of this message, cross-link 90' sends the 
contents of Its serial control and status register 
912' back to cross-link 90. 

75 Fig. 1 1 shows some of the elements for routing 
control and status signals (referred to as "control 
codes") in primary cross-link 90 and mirror cross- 
link 95. Corresponding cross-link elements exist in 
the preferred embodiment within cross-links 90' 

20 and 95 . These codes are sent between the mem- 
ory controllers 70 and 75 and the I/O modules 
coupled to module interconnects 130, 132. 130' 
and 132 

Rg. 12 shows the elements In the preferred 

25 embodiment^ of primary cross-link 90 which are 
used for routing data and address signals. Cor- 
responding cross-link elements exist in cross-links 
95, 90' and 95'. 

In Rg. 11, the elements for both the primary 

30 cross-link 90 and mirror cross-link 95 in processing 
system 20 are shown, although the hardware Is 
identical, because of an Important interconnection 
between the elements. The circuit elements in mir- 
ror cross-link 95 which are equivalent to elements 

35 In primary cross-link 90 are shown by the same 
number, except in the mirror controller the letter 
"m" is placed after the number. 

With reference to Rgs. 11 and 12, the ele- 
ments include latches, multiplexers, drivers and 

40 receivers. Some of the latches, such as latches 933 
and 933m, act as delay elements to ensure the 
proper timing through the cross-links and thereby 
maintain synchronization. As shown in Rg. 1 1 , con- 
trol codes from memory controller 70 are sent via 

45 bus 88 to latch 931 and then to latch 932. The 
reason for such latching is to provide appropriate 
delays to ensure that data from memory controller 
70 passes through cross-link 90 simultaneously 
with data from memory controller 70'. 

so If codes from memory controller 70 are to be 
sent to processing system 20' via cross-link 90', 
then driver 937 is enabled. The control codes from 
memory controller 70 also pass through latch 933 
and into multiplexer CSMUXA 935. If control codes 

55 are received into primary cross-link 90 from cross- 
link 90'. then their path is through receiver 936 into 
latch 938 and also into multiplexer 935. 

Control codes to multiplexer 935 determine the 
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source of data, that is either from memory control- 
ler 70 or from memory controller 70', and place 
' those codes on the output of multiplexer 935. That 
output is stored in latch 939. again. for proper delay 
purposes, and driver 940 Is enabled if the codes 
are to be sent to module interconnect 130. 

The path for data and address signals, as 
shown in Fig, 1 2 is somewhat similar to the path of 
control signals shown in Fig. 11. The differences 
reflect the fact that during any one transaction, data 
and addresses are flowing in only one direction 
through cross-links 90 and 95, but control signals 
can be flowing in both directions during that trans- 
action. For that same reason the data lines in 
busses 88 and 89 are bidirectional, but the control 
codes are not. 

Data and addresses from the memory control- 
ler 70. via bus 88. enter latch 961, then latch 962, 
and then latch 964. As in Fig. 11, the latches in 
Fig. 12 provide proper timing to maintain synchro- 
nization. Data from memory controller 70' is buf- 
fered by receiver 986, stored In latch 988. and then 
routed to the input of multiplexer MUXA 966. The 
output of multiplexer 966 is stored in latch 968 and. 
if driver 969 is enabled, is sent to module intercon- 
nect 130. 

The path for control codes to be sent to mem- 
ory controller 70 is shown in Fig, 11. Codes from 
module interconnect 130 are first stored in . latch 
941 and then presented to multiplexer CSMUXC 
942. Multiplexer 942 also receives control codes 
from parallel cross-link registers 910 and selects 
either the parallel register codes or the codes from 
latch 941 for transmission to latch 943. If those 
control codes are to be transmitted to cross-link 
90', then driver 946 is enabled. Control codes from 
cross-link 90' (and thus from memory controller 
70 ) are buffered by receiver 947, stored in latch 
948. and presented as an input to multiplexer 
CSMUXD 945. CSMUXD 945 also receives as an 
input the output of latch 944 which stores the 
contents of latch 943. 

Multiplexer 945 selects either the codes from 
module interconnect 130 or from cross-link 90 and 
presents those signals as an input to multiplexer 
CSMUXE 949. Multiplexer 949 also receives as 
inputs a code from the decode logic 970 (for bulk 
memory transfers that occur during resynchroniza- 
tion), codes from the serial cross-link registers 920, 
or a predetermined error code ERR. Multiplexer 
949 then selects ones of those Inputs, under the 
appropriate control, for storage in latch 950. If 
those codes are to be sent to memory controller 
70, then driver 951 is activated. 

The purpose of the error code ERR, which Is 
an input into multiplexer 949, is to ensure that an 
error in one of the rails will not cause the CPUs in 
the same zone as the rails to process different 



Inforrhation. If this occurred, CPU module 30 would 
detect a fault which would cause drastic, and per- 
haps unnecessary action. To avoid this, cross-link 
90 contains an EXCLUSIVE OR gate 960 which 

5 compares the outputs of multiplexers 945 and 
945m. If they differ, then gate 960 causes mul- 
tiplexer 949 to select the ERR code. EXCLUSIVE 
OR gate 960m similarly causes multiplexer 949m 
also to select an ERR code. This code indicates to 

10 memory controllers 70 and 75 that there has been 
an enror, but avoids causing a CPU module error. 
The single rail interface to memory module 60 
accomplishes the same result for data and ad- 
dresses. 

75 The data and address flow shown in Rg. 12 Is 
similar to the flow of control signals in Fig. 11. Data 
and addresses from module interconnect 130 are 
stored in latch 972 and then provided as an input 
to multiplexer MUXB 974. Data from the parallel 

20 registers 910 provide another input to multiplexer 
974. The output of multiplexer 974 is an Input to 
multiplexer MUXC 976 which also receives data 
and addresses stored in latch 961 that were 
originally sent from memory controller 70. Mul- 

25 tiplexer 976 then selects one of the inputs for 
storage in latch 978. If the data and addresses, 
either from the module interconnect 130 or from 
the memory controller 70. are to be sent to cross- 
link 90'. then driver 984 is enabled. 

30 Data from cross-link 90 is buffered by receiver 

986 and stored in latch 988, which also provides an 
input to multiplexer MUXD 982. The other input of 
multiplexer MUXD 982 is the output of latch 980 
which contains data and addresses from latch 978, 

35 Multiplexer 982 then selects one of its inputs which 
is then stored into latch 990. If the data or ad- 
dresses are to be sent to memory controller 70, 
then driver 992 is activated. Data from serial regis- 
ters 920 are sent to memory controller 70 via driver 

40 994. 

The data routing in cross-link 90, and more 
particularly the control elements in both Figs. 11 
and 12. is controlled by several signals generated 
by decode logic 970, decode logic 971. decode 

45 logic 996, and decode logic 998. This logic pro- 
vides the signals which control multiplexers 935. 
942. 945, 949, 966, 974, 976, and 982 to select the 
appropriate input source. In addition, the decode- 
logic also controls drivers 940, 946, 951 , 969, 984, 

50 992, and 994. , 

Most of the control signals are generated by 
decode logic 998. but some are generated by 
decode logic 970. 971, 970m, 971m, and 996. 
Decode logic 998, 970 and 970m are connected at 

55 positions that will ensure that the logic will receive 
the data and codes necessary for control whether 
the data and codes are received from its own zone 
or from other zone. 
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The purpose of decode iogic 971, 971m and 
996 is to ensure that the drivers 937, 937m and 
984 are set Into the proper state. This "early de- 
code" makes sure that data addresses and codes 
will be forwarded to the proper cross-links in all 5 
cases. Without such early decode logic, the cross- 
links could all be in a state with their drivers 
disabled. If one at the memory controllers were 
also disabled, then Its cross-links would never re- 
ceive addresses, data and control codes, effec- io 
tively disabling all the I/O modules connected to 
that cross-link. 

Prior to describing the driver control signals 
generated by decode logic 970. 971, 970m, 971m, 
and 998, It Is necessary to understand the different 7S 
modes that these zones, and therefore the cross- 
links 90 and 95. can be in. Fig. 13 contains a 
diagram of the different states A-F, and a table 
explaining the states which correspond to each 
mode. 20 

At start-up and in other instances, both zones 
are in state A which is known as the OFF mode for 
both zones. In that mode, the computer systems In 
both zones are operating independently. After one 
of the zones' operating system requests the ability 25 
to communicate with the I/O of the other zone, and 
that request is honored, then the zones enter the 
master/slave mode, shown as states B and C. In 
such modes, the zone which Is the master, has an 
operating CPU and has control of the I/O modules .30 
of its zone and of the other zone. 

Upon initiation of resynchronizatlon, the com- 
puter system leaves the master/slave modes, either 
states B or C, and enters a resync slave/resync 
master mode, which is shown as states E and F. In 35 
those modes, the zone that was the master zone is 
in charge of bringing the CPU of the other zone on 
line. If the resynchronizatlon fails, the zones revert 
to the same master/slave mode that they were In 
prior to the resynchronizatlon attempt. 40 

If the resynchronizatlon is successful, however, 
then the zones enter state D. which Is the full 
duplex mode. In this mode, both zones are operat- 
ing together in lockstep synchronization. Operation 
continues in this mode until there Is a CPU/MEM 45 
fault, in which case the system enters one of the 
two master/slave modes. The slave is the zone 
whose processor experienced the CPU/MEM fault 

When operating in state D, the full duplex 
mode, certain en-ors, most notably clock phase so 
errors, necessitate splitting the system Into two 
independent processing systems. This causes sys- 
tem 10 to go back into state A. 

Decode logic 970, 970m, 971. 971m, and 998 
(collectively refenred to as the cross-link control 55 
logic), which are shown in Rgs. 11 and 12. have 
access to the resync mode bits 915 and the cross- 
link mode bits 916, which are shown in Rg. 10. in 
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' order to determine how to set the cross-link drivers 
and multiplexers into the proper states. In addition, 
the cross-link decode logic also receives and ana- 
lyzes a portion of an address sent from memory 
controllers 70 and 75 during data transactions to 
extract addressing Information that further indicates 
to the cross-link decode logic how to set the state 
of the cross-link multiplexers and drivers. 

The information needed to set the states of the 
multiplexers is fairiy straightforward once the dif- 
ferent modes and transactions are understood. The 
only determination to be made is the source of the 
data. Thus when cross-links 90 and 95 are In the 
slave mode, multiplexers 935, 935m, and 966 will 
select data addresses and codes from zone II'. 
Those multiplexers will also select data, addresses- 
and codes from the other zone If cross-links 90 and 
95 are in full duplex mode, the address of an 1/0 
Instruction is for a device connected to an I/O 
module In zone 11, and the cross-link with the 
affected multiplexer is in a cross-over mode. In a 
cross-over mode, the data to be sent on the niod- 
ule Interconnect is to be received from the other 
zone for checking. In the preferred embodiment, 
module interconnect 130 would receive data, ad- 
dresses and codes from the primary rail in zone 1 1 
and nriodule Interconnect 'would receive data, ad- 
dresses and codes from the mirror rail in zone I1'. 
Alternatively, module interconnect 132 could re- 
ceive data, addresses and codes from the primary 
rail in zone 11 which would allow the primary rail 
of one zone to be compared with the min'or rail of 
the other zone. 

Multiplexers 945, 945m, and 982 will be set to 
accept data, address and codes from whichever 
zone is the source of the data. This is true both 
when all the cross-links are in full duplex mode and 
the data, address and codes are received from I/O 
modules and when the cross-link is in a resync 
slave mode and the data, address and codes are 
received from the memory controllers of the other 
zone. 

If the addressing information from memory 
controllers 70 and 75 indicates that the source of 
response data and codes is the cross-link's own 
parallel registers 910. then multiplexers 942, 942m, 
and 974 are set to select data and codes from 
those registers. Similarly, if the addressing informa- 
tion from memory controllers 70 and 75 indicates 
that the source of response data Is the cross-link's 
own serial register 920. then multiplexers 949 and 
949m are set to select data and codes from those 
registers. 

Multiplexers 949 and 949m are also set to 
select data from decode logic 970 and 970m. re- 
spectively, if the Information is a control code dur- 
ing niemory resync operations, and to select the 
ERR code if the EXCLUSIVE OR gates 960 and 
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960m identify a miscompare between the data 
transmitted via cross-[inl<s 90 and 95. In this latter 
case, the control of the multiplexers 949 and 949m 
is generated from the EXCLUSIVE OR gates 960 
and 960m rather than from the cross-link control 
logic. Multiplexers 949 and 949m also select codes 
from serial cross-link registers 910 when those 
registers are requested or the output of multiplex- 
ers 945 and 945m when those codes are request- 
ed. Multiplexers 945 and 945m select either the 
outputs from multiplexers 942 and 942m, respec- 
tively, or I/O codes from cross-links 90' and 95', 
respectively. 

Multiplexer 976 selects either data and ad- 
dresses from module interconnect 130 in the case, 
of a transaction with an I/O module, or data and 
addresses from memory controller 90 when the 
data and addresses are to be sent to cross-link 90' 
either for. I/O or during memory resynchronization. 

Drivers 937 and 937m are activated when 
cross-links 90 and 95 are in duplex, master or 
resync master modes. Drivers 940 and 940m are 
activated for I/O transactions in zone 11. Drivers 
946 and 946m are activated when cross-links 90 
and 95 are in the duplex or slave modes. Drivers. 
951 and 951m are always activated. 

Driver 969 is activated during I/O writes to zone 
11. Driver 984 is activated when cross-link 90 Is 
sending data and addresses to I/O in zone 1l', or 
when cross-link 90 Is in the resync master mode. 
Receiver 986 receives data from cross-link 90'. 
Drivers 992 and 994 are activated when data is 
being sent to memory controller 70; driver 994 is 
activated when the contents of the serial cross-link 
register 910 are read and driver 992 is activated 
during ail other reads. 



5. Oscillator 

When both processing systems 20 and 20' are 
each performing the same functions in the full 
duplex mode, it is imperative that CPU modules 30 
and 30' perform operations at the same rate. Oth- 
erwise, massive amounts of processing time will be 
consumed in resynchronizing processing systems. 
20 and 20' for I/O and interprocessor error check- 
ing. In the preferred embodiment of processing 
systems 20 and 20'. their basic clock signals are 
synchronized and phase-locked to each other. The 
fault tolerant computing system 10 includes a tim- 
ing system to control the frequency of the clock 
signals to processing systems 20 and 20' and to 
minimize the phase difference between the clock 
signals for each processing system. 

Fig. 14 shows a block diagram of the timing 
system of this invention embedded in processing 
systems 20 and 20 . The timing system comprises 
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oscillator system 200 in CPU module 30 of pro- 
cessing system 20, and oscillator system 200' In 
CPU module 30' of processing system 20'. The 
elements of oscillator 200' are equivalent to those 

6 for oscillator 200 and both oscillator systems' op- 
eration is the same. Thus, only the elements and 
operation of oscillator system 200 will be de- 
scribed, except if the operations of oscillator sys- 
tems 200 and 200' differ. 

10 As Fig. 14 shows, much of oscillator system 
200, specifically the digital logic, lies inside of 
cross-link 95, but that placement is not required for 
the present invention. Oscillator system 200 in- 
cludes a voltage-controlled crystal oscillator 

75 (VCXO) 205 which generates a basic oscillator sig- 
nal preferably at 66.66 MHz. The frequency of 
VCXO 205 can be adjusted by the voltage level at 
the input. 

Clock distribution chip 210 divides down the 
20 basic oscillator signal and preferably produces four 
primary clocks all having the same frequency. For 
primary CPU 40 the clocks are PCLK L and PCLK 
H, which are logical inverses of each other. For 
mirror CPU 50, clock distribution chip 210 pro- 
26 duces clock signals MCLK L and MCLK H, which 
are also logical inverses of each other. The timing 
and phase relationship of these clock signals are 
shown In Fig. 15. Preferably, frequency of clock 
signals PCLK L. PCLK H. MCLK L, and MCLK H is 
30 about 33.33 MHz. Clock chip 210 also produces a 
phase-locked loop signal CLKC H at 16.66 MHz, 
also shown In Fig, 15. This phase locked loop 
signal is sent to clock logic 220 which buffers that 
signal. 

35 Clock logic buffer 220 sends the CLKC H sig- 

nal to oscillator 200' for use' in synchronization. 
Clock logic buffer 220' In oscillator 200' sends Its 
own buffered phase-locked loop signal CLKC H to 
phase detector 230 in oscillator 200. Phase, detec- 

40 tor 230 also receives the buffered phase locked 
loop signal CLKC H from clock logic 220 through 
delay element 225. Delay element 225 approxi- 
mates the delay due to the cable run from clock 
logic buffer 220 . 

45 Phase detector 230 compares its input phase 
locked loop signals and generates two outputs. 
One is a phase differences signal 235 which is sent 
through loop amplifier 240 to the voltage input of 
VCXO 205. Phase differences signal 235 will cause 

50 amplifier 240 to -generate a signal to alter the 
frequency of VCXO 205 to compensate for phase 
differences. 

The other output of phase detector 230 is a 
phase error signal 236 which indicates possible 
55 synchronism faults. 

Rg. 16 is a detailed diagram of phase detector 
230. Phase detector 230 includes a phase com- 
parator 232 and a voltage comparator 234. Phase 
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comparator. 232 receives the clock signal from de- 
lay element 225 (CLKC H) and the phase lock loop 
dock signal from oscillator 20o' (CLKC H) and 
generates phase differences signal 235 as a volt- 
age level representing the phase difference of s 
those signals. 

If processing system 20 were the "slave" for 
purposes' of clock synchronization, switch 245 
would be in the "SLAVE" position (i.e., closed) and 
the voltage level 235, after being amplified by loop to 
amplifier 240, would control the frequency of VCXO 
205. If both switches 245 and 245' are in the 
"master" position, processing systems 20 and 20' 
would not be phase-locked and would be running 
asynchronously (Independently). is 

The voltage level of phase differences signal 
235 is also an input to voltage comparator 234 as . 
are two reference voltages, Vrefi and Vreiz. repre- 
senting acceptable ranges of phase lead and lag. If 
the phase difference Is within tolerance, the PHASE 20 
ERROR signal will not be activated. If the phase 
difference is out of tolerance, then the PHASE 
ERROR signal 236 will be activated and sent to 
cross-link 95 via clock decoder 220. 



6. I/O Module 

Fig. 17 shows a prefenred embodiment of an 
1/0 module 100. The principles of operation 1/0 30 
module 100 are applicable to the other 1/0 modules 
as well. 

Fig. 18 shows the elements in the preferred 
embodiment of firewall 1000. Firewall 1000 In- 
cludes a 16 bit bus interface 1810 to module 35 
interconnect 130 and a 32 bit bus interface 1820 
for connection to bus 1020 shown in Rg. 17. Inter- 
faces 1810 and 1820 are connected by an internal 
firewall bus 1815 which also Interconnects with the 
other elements of firewall 1000. Preferably bus 40 
1815 is a parallel bus either 16 or 32 bits wide. 

I/O module 100 is connected to CPU module 
30 by means of dual rail module interconnects 130 
and 132. Each of the module interconnects is re- 
ceived by firewalls 1000 and 1010. respectively. 45 
One of the firewalls, which is usually, but not al- 
ways firewall 1000. writes the data from module 
interconnect 130 onto bus 1020. The other firewall, 
in this case firewall 1010, checks that data against 
its own copy received from module interconnect so 
132 using firewall comparison circuit 1840 shown in 
Rg. 18. That checking is effective due to the lock- 
step synchronization of CPU modules 30 and 30' 
which causes data written to I/O module 100 from 
CPU modules 30 and 30' to be available at 55 
firewalls 1000 and 1010 substantially simultaneous- 
ly. 

Rrewall comparison circuit 1840 only checks 



data received from CPU modules 30 and 30', Data 
sent to CPU modules 30 and 30' from an I/O 
device have a common origin and thus do not 
require checking. Instead, data received, from an 
I/O device to be sent to CPU modules 30 and 30' 
is checked by an error detection code (EDC), such 
as a cyclical redundancy check (CRC). which is 
performed by EDC/CRC generator 1850. EDC/CRC 
generator 1850 is also coupled to Internal firewall 
bus 1815. 

EDC/CRC generator 1850 generates and 
checks the same EDC/CRC code that is used by 
the I/O device. Preferably, I/O module 100 gen- 
erates two EDC. One, which can also be a 
EDC/CRC, is used for an interface to a network, 
such as tiie Ethernet packet network to which mod- 
ule 100 Is coupled (see element 1082 in Fig. 17), 
The other is used for a disk Interface such as disk 
interface 1072 in Rg. 17. 

EDC/CRC coverage Is not required between 
CPU module 30 and I/O module 100 because the 
module interconnects are duplicated. For example 
in . CPU module 30, cross-link 90 communicates 
witii firewall 1000 through module interconnect 130, 
and cross-link 95 communicates with firewall 1010 
through module interconnect 132. 

A message received from Ethernet network 
1082 is checked for a valid EDC/CRC by network 
control 1080 shown in Rg. 17. The data, complete 
with EDC/CRC, is written to a local RAM 1060 also 
shown in Fig. 17. All data in local RAM 1060 is 
transfen^ed to memory module 60 using DMA. A 
DMA control 1890 coordinates the transfer and 
directs EDC/CRC generator 1850 to check the va- 
lidity of the EDC/CRC encoded data being trans- 
ferred. 

Most data transfers with an 1/0 device are done 
with DMA. Data is moved between main memory 
and 1/0 buffer memory. When data is moved from 
the main memory to an I/O buffer memory, an 
EDC/CRC may be appended. When the data is 
moved from I/O buffer memory to main memory, 
an EDC/CRC may be checked and moved to main 
memory or may be stripped. When data is moved 
from the I/O buffer memory through an external 
device, such as a disk or Ethernet adaptor the 
EDC/CRC may be checked locally or at a distant 
receiving node, or both. The memory data packets 
may have their EDC/CRC generated at the distant 
node or by the local Interface on tiie I/O module. 

This operation ensures that data residing in or 
being transferred through a single rail system like 
I/O module 100 Is covered by an enror detection 
code, which is preferably at least as reliable as the 
communications media the data will eventually 
pass through. Different I/O modules, for example 
those which handle synchronous protocols, prefer- 
ably have an EDC/CRC generator which generates 
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and checks the EDC/CRC codes of the appropriate 
protocols. 

In general, DMA control 1890 handles the por- 
tion of a DMA operation specific to the shared 
memory controller 1050 and local RAM 1060 being 
addressed. The 32 bit bus 1020 is driven in two 
different modes. During DMA setup, DiyiA control 
1890 uses bus 1020 as a standard asynchronous 
microprocessor bus. The address in local RAM 
1060 where the DMA operation will occur Is sup- 
piled by shared memory controller 1050 and DMA 
control 1890. During the actual DMA transfer. DMA 
control 1890 directs DMA control lines 1895 to 
drive bus 1020 in a synchronous fashion. Shared 
memory controller 1050 will transfer a 32 bit data 
word with bus 1020 every bus cycle, and DMA 
control 1890 keeps track of how many words are 
left to be transferred. Shared memory control 1050 
also controls local RAM 1060 and creates the next 
DMA address. 

The 1/0 modules (100, 110. 120) are responsi- 
ble for controlling the read/write operations to their 
own local RAM 1060. The CPU module 30 Is 
responsible for controlling the transfer operations 
with memory array 60. The DMA engine 800 of 
memory controllers 70 and 75 (showri In Fig. 8) 
directs the DMA operations on the CPU module 30. 
This division of labor prevents a fault in the DMA 
logic on any module from degrading the data integ- 
.rity on any other module In zones 11 or 1 1 . 

The functions of trace RAM 1 872 and trace 
RAM controller 1870 are described in greater detail 
below. Briefly, when a fault is detected and the 
CPUs 40, 40', 50 and 50' and CPU modules 30 
and 30 are notified, various trace RAMs throughout 
computer system 10 are caused to perform certain 
functions described below. The communications 
with the trace RAMs takes place over trace bus 
1095. Trace RAM control 1870, In response to 
signals from trace bus 1095, causes trace RAM 
1872 either to stop storing, or to dump its contents 
over trace bus 1095. 

I/O module bus 1020, which is preferably a 32 
bit parallel bus. couples to firewalls 1000 and 1010 
as well as to other elements of the I/O module 100. 
A shared memory controller 1050 is also coupled 
to I/O bus 1020 in I/O module 100. Shared memory 
controller 1050 is coupled to a local memory 1060 
by a shared memory bus 1065, which preferably 
carries 32 bit data. Preferably, local memory 1060 
is a RAM with 256 Kbytes of memory, but the size 
of RAM 1060 is discretionary. The shared memory 
controller 1050 and local RAM 1060 provide mem- 
ory capability for I/O module 100. 

Disk controller 1070 provides a standard inter- 
face to a disk, such as disks 1075 and 1075' in Fig. 
1. Disk controller 1070 is also coupled to shared 
memory controller 1050 either for use of local RAM 



1060 or for communication with I/O module bus 
1020. 

A network controller 1080 provides an interface 
to a standard network, such as the ETHERNET 

5 network, by way of network interface 1082. Network 
controller 1080 is also coupled to shared memory 
controller 1050 which acts as an interface both to 
local RAM 1060 and I/O module bus 1020. There is 
no requirement, however, for any one specific or- 

10 ganization or structure of I/O module bus 1020. 

PCIM (power and cooling interface module) 
support element 1030 is connected to 1/0 module 
bus 1020 and to an ASCII interface 1032. PCIM 
support element 1030 allows processing system 20 

IS to monitor the status of the power system (i.e.. 
batteries, regulators, etc.) and the cooling system 
(i.e.. fans) to ensure their proper operation. Prefer- 
ably, PCIM support element 1030 only receives 
messages when there is some fault or potential 

20 fault indication, such as an unacceptably low bat- 
tery voltage. It is also possible to use PCIM sup- 
port element 1030 to monitor all the power and 
cooling subsystems periodically. Alternatively PCIM 
support element 1030 may be connected directly 

25 to firewalls 1000 and 1010. 

Diagnostics microprocessor 1100 is also con- 
nected to the I/O module bus 1020. In general, 
diagnostics microprocessor 1100 is used to gather 
error checking information from trace RAMS, such 

30 as trace RAM 1872, when faults are detected. That 
data is gathered into trace buses 1095 and 1096. 
through firewalls 1000 and 1010, respectively, 
through module bus 1020, and Into microprocessor 
1100. 

35 

D. INTERPROCESSOR AND INTERMODULE 
COMMUNICATION 

40 

i. Data Paths 

The elements of computer system 10 do not 
by themselves constitute a fault tolerant system, 

45 There needs to be a communications pathway and 
protocol which allows communication during normal 
operations and operation during fault detection and 
correction. Key to such communication is cross-link 
pathway 25. Cross-link pathway 25 comprises the 

so parallel links, serial links, and clock signals already 
described. These are shown in Rg. 19. The parallel 
link includes two identical sets of data and address 
lines, control lines. Intenrupt lines, coded error 
lines, and a soft reset request line. The data and 

65 address lines and the control lines contain informa- 
tion to be exchanged between the CPU modules, 
such as from the module interconnects 130 and 
132 (or 130' and 132) or from memory module 60 
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(60'). 

The interrupt lines preferably contain one line 
for each of the interrupt levels available to 1/0 
subsystem (modules 100. 110, 120, lOO', 110' and 
120 ). These lines are shared by cross-links 90. 95, 
90' and 95'. 

The coded error lines preferably include codes 
for synchronizing a console "HALT" request for 
both zones, one for synchronizing a CPU error for 
both zones, one for indicating the occurrence of a 
CPU/memory failure to the other zone, one for 
synchronizing DMA en-or for both zones, and one 
for indicating clock phase error. The error lines 
from each zone 11 or 1l' are inputs to an OR gate, 
such as OR gate 1990 for zone 11 or OR gate 
1990' for zone 1l'. The output at each OR gate 
provides an input to the cross-links of the other 
zone. 

The fault tolerant processing system 10 Is de- 
signed to continue operating as a dual rail system 
despite transient faults. The I/O subsystem 
(modules 100, 110, 120. 100', 110', 120') can also 
experience transient enrors or faults and continue to 
operate. In the prefen-ed embodiment, an enror 
detected by firewall comparison circuit 1840 will 
cause a synchronized error report to be made 
through pathway 25 for CPU directed operations. 
Hardware In CPU 30 and 30' will cause a synchro- 
nized soft reset through pathway 25 and will retry 
the faulted operation. For DMA directed operations, 
the same error detection results in synchronous 
intenojpts through^ pathway 25, and software in 
CPUs 40, 50, 40' and 50' will restart the DMA 
operation. 

Certain transient errors are not immediately 
recoverable to allow continued operation in a full- 
duplex, synchronized fashion. For example, a con- 
trol error in memory module 60 can result in un- 
known data In memory module 60. In this situation, 
the CPUs and memory elements can no longer 
function reliably as part of a fail safe system so 
they are* removed. Memory array 60 must then 
undergo a memory resync before the CPUs and 
memory elements can rejoin the system. The 
CPU/memory fault code of the coded error lines in 
pathway 25 indicates to CPU 30' that the CPUs 
and memory elements of CPU 30 have been fault- 
ed. 

The control lines, which represent a combina- 
tion of cycle type, error type, and ready conditions, 
provide the handshaking between CPU modules 
(30 and 30') and the I/O modules. Cycle type, as 
explained above, defines the type of bus operation 
being performed: CPU I/O read. DMA transfer. DMA 
setup, or interrupt vector request. Error type de- 
fines either a firewall miscompare or a CRC error. 
"Ready" messages are sent between the CPU and 
I/O modules to indicate the completion of request- 
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ed operations. 

The serial cross-link includes two sets of two 
lines to provide a serial data transfer for a status 
read, ioopback, and data transfer. 
5 The clock signals exchanged are the phase 
locked clock signals CLKC H and CLKC' H 
(delayed). 

Figs. 20A-D show block diagrams of the ele- 
ments of CPU modules 30 and 30' and I/O mod- 

70 ules 100 and lOO' through which data passes dur- 
ing the different operations. Each of those elements 
has each been described previously. 

Fig. 20A shows the data pathways for a typical 
CPU I/O read operation of data from an 1/0 module 

75 1 00, such as a CPU I/O register read operation of 
register data from shaved memory controller 1050 
(1O50'). Such an operation will be referred to as a 
read of local data, to distinguish it from a DMA 
read of data from local memory 1060, which usu- 

20 ally contains data from an Internal device controller. 
The local data are presumed to be stored in local 
RAM 1060 (1060') for transfer through shared 
memory controller 1050 (1050'). For one path, the 
- data pass through firewall 1000, module intercon- 

25 nect 130, to cross-link 90. As seen In Rg. 12, 
cross-link 90 delays the data from firewall 1000 to 
memory controller 70 so that the data to cross-link 
90 may be presented to memory controller 70 at 
the same time the data are presented to memory 

30 controller 70, thus allowing processing systems 20 
and . 20 to remain synchronized. The data then 
proceed out of memory controllers 70 and 70' into 
CPUs 40 and 40 by way of internal busses 46 and 
46'. 

35 A similar path is taken for reading data into 
CPUs 50 and 50'. Data from the shared memory 
controller 1050 proceeds through firewall 1010 and 
into cross-link 95. At that time, the data are routed 
both to cross-link 95 and through a delay unit 

40 inside cross-link 95. 

CPU I/O read operations may also be per- 
formed for data received from the I/O devices of 
processing system 20' via a shared memory con- 
troller 1050' and local RAM in I/O device 100 , 

45 Although I/O modules 100, 110, and 120 are 
similar and correspond to I/O modules IOO', IIO', 
and 120', respectively, the corresponding I/O mod- 
ules are not in lockstep synchronization. Using 
memory controller 1050' and local RAM IO60' for 

50 CPU I/O read, tiie data would first go to cross-links 
90 and 95 . The remaining data path is equivalent 
to the path from memory controller 1050. The data 
travel from the cross-links 90' and 95* up through 
memory controllers 70' and 75' and finally to CPUs 

55 40 and 50', respectively. Simultaneously, tiie data 
travel across to cross-links 90 and 95, respectively, 
and then, without passing tiirough a delay element, 
the data continue up to CPUs 40 and 50, respec- 
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tively. 

Rg. 20B shows a CPU I/O write operation of 
local data. Such local data are transferred from the 
CPUs 40, 50, 40' and 50' to an I/O module, such 
as I/O module 100. An example of such an opera- 
tion is a write to a register in shared memory 
controllers 1050. The data transferred by CPU 40 
proceed along the same path but in a direction 
opposite to that of the data during the CPU I/O 
read. Specifically, such data pass through bus 46. 
memory controller 70. various latches (to permit 
synchronization), firewall 1000, and memory con- 
troller 1050. Data from CPU 50' also follow the path 
of the CPU I/O reads in a reverse direction. Specifi- 
cally, such data pass through bus 56 , memory 
controller 75 , cross-link 95 , cross-link 95, and into 
firewall 1010. As indicated above, firewalls 1000 
and 1010 check the data during I/O write oper- 
ations to check for errors prior to storage. 

When writes are performed to an I/O module in 
the other zone, a similar operation is performed. 
However, the data from CPUs 50 and 40 . are used 
instead of CPUs 50' and 40. 

The data from CPUs 50 and 40' are transmit- 
ted through symmetrical paths to shared memory 
controller 1050'. The data from CPUs 50 and 40' 
are compared by firewalls lOOO' and lOlO'. The 
reason different CPU pairs are used to service I/O 
write data is to allow checking of all data paths 
during normal use in a full duplex system. Interrail 
checks for each zone were previously performed at 
memory controllers 70, 75, 70' and 75'. 

Fig. 20C shows the data paths for DMA read 
operations. The data from memory array 600 pass 
simultaneously into memory controllers 70 and 75 
and then to cross-links 90 and 95. Cross-link 90 
delays the data transmitted to firewall 1000 so that 
the data from cross-links 90 and 95 reach firewalls 
1000 and 1010 at substantially the same time. 

Similar to the CPU 1/6 write operation, there 
are four copies of data of data to the various cross- 
links. At the firewall, only two copies are received. 
A different pair of data are used when performing 
reads to zone 1 1 . The data paths for the DMA write 
operation are shown in Fig. 20D and are similar to 
those for a CPU I/O read. Specifically, data from 
shared memory controller. 1050' proceed through 
firewall 1000 . cross-link 90 (with a delay), memory 
controller 70 . and into memory array 600'. Si- 
multaneously, the data pass through firewall 1010', 
cross-link 95 (with a delay), and memory controller 
75 , at which time it is compared with the data from 
memory controller 70 during an interrail error 
check. As with the CPU I/O read, the data in a 
DMA write operation may alternatively be brought 
up through shared memory controller 1050 in an 
equivalent operation. 

The data out of cross-link 90' also pass 
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through cross-link 90 and memory controller 70 
and into memory anray 800. The data from cross- 
link 95 pass through cross-link 95 and memory 
controller 75, at which time they are compared with 

5 the data from memory controller 70' during a si- 
multaneous interrail check. 

The data path for a memory resynchronization 
(resync) operation is shown in Fig. 20E. In this 
operation the contents of both memory arrays 60 

70 and 60' must be set equal to each other. In mem- 
ory resync, data from memory array 600' pass 
through memory controllers 70' and 75' under 
DMA control, then through cross-links 90' and 95', 
respectively. The data then enters cross-links 90 

75 and 95 and memory controllers 70 and 75, respec- 
tively, before being stored in memory array 600. 



2. Resets 

20 

The preceding discussions of system 10 have 
made reference to many different needs for resets. 
, In certain instances not discussed, resets are used 
for standard functions, such as when power is 

25 initially applied to system 10. Most systems have a 
single reset which always sets the processor back 
to some predetermined or initial state, and thus 
disrupts the processors' instruction flow. Unlike 
most other systems, however, resets in system 10 

30 do not affect the flow of Instruction execution by 
CPUs 40, 40', 50 and 50' unless absolutely neces- 
sary. In addition, resets in system 10 affect only 
those portions that need to be . reset to restore 
normal operation. 

35 Another aspect of the resets in system 10 is 

their containment. One of the prime considerations 
in a fault tolerant system is that no function should 
be allowed to stop the system from operating 
should that function fail. For this reason, no single 

40 reset in system 10 controls elements of both zones 
11 and 1l' without direct cooperation between 
zones 11 and 1l'. Thus, In full duplex mode of 
operation, all resets in zone 11 will be independent 
of resets in zone 1l'. When system 10 is in 

45 master/slave mode, however, the slave zone uses 
the resets of the master zone. In addition, no reset 
in system 10 affects the contents of memory chips. 
Thus neither cache memory 42 and 52, scratch 
pad memory 45 and 55 nor memory module 60 

50 lose any data due to a reset. 

There are preferably three classes of resets in 
system 10; "clock reset," "hard reset," and "soft 
reset." A clock reset realigns all the clock phase 
generators in a zone. A clock reset in zone 1 1 will 

55 also Initialize CPUs 40 and 50 and memory module 
60. A clock reset does not affect the module inter- 
connects 130 and 132 except to realign the clock 
phase generators on those modules. Even when 
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system 10 is in master/slave mode, a clock reset in 
the slave zone will not disturb data transfers from 
the master zone to the slave zone module intercon- 
nect. A clock reset in zone 1l', however, will Initial- 
ize the corresponding elements in zone 1l'. 

In general, a hard reset returns all state "de- 
vices and registers to some predetermined or initial 
state, A soft reset only returns state engines and 
temporary storage registers to their predetermined 
or initial state. The state engine in a module is the 
circuitry that defines the state of that module. Reg- 
isters containing error information and configuration 
data will not be affected by a soft reset. Addition- 
ally, system 10 will selectively apply both hard 
resets and soft resets at the same time to reset 
only those elements that need to be reinitialized in 
order to continue processing. 

The hard resets clear system 10 and. as in 
conventional systems, return system 10 to a known 
configuration. Hard resets are used after power is 
applied, when zones are to be synchronized, or to 
initialize or disable an I/O module. In system 10 
there are preferably four hard resets: "power up 
reset." "CPU hard reset," "module reset," and 
"device reset." Hard resets can be further broken 
down into local and system hard resets. A local 
hard reset only affects logic that responds when 
the CPU is in the slave mode. A system hard reset 
is limited to the logic that is connected to cross-link 
cables 25 and module interconnects 130 and 132. 

The power up reset is used to initialize zones 
11 and 11 ' Immediately after power Is supplied. 
The power up reset forces an automatic reset to all 
parts of the zone. A power up reset Is never 
connected between the zones of system 1 1 be- 
cause each zone has its own power supply and will 
thus experience different length "power-on" events. 
The power up reset is implemented by applying all 
hard resets and a clock reset to zone 11 or 1 1 

The CPU hard reset is used for diagnostic ' 
purposes in order to return a CPU module to a 
known state. The CPU hard reset clears all in- 
formation in the CPUs, memory controllers, and 
memory module status registers in the affected 
zone. Although the cache memories and memory 
modules are disabled, the contents of the scratch 
pad RAMs 45 and 55 and of the memory module 
60 are not changed. In addition, unlike the power 
up reset, the CPU hard reset does not modify the 
zone Identification of the cross-links nor the clock 
mastership. The CPU hard reset is the sum of all 
local hard resets that can be applied to a CPU 
module and a clock reset. 

The module hard reset is used to set the I/O 
modules to a known state, such as during boot- 
strapping, and is also used to remove a faulting I/O 
module from the system. The I/O module hard 
reset clears everything on the I/O module, leaves 



. the firewalls in a diagnostic mode, and disables the 
drivers. 

A device reset is used to reset I/O devices 
connected to the I/O modules. The resets are de- 

5 vice dependent and are provided by the I/O mod- 
ule to which the device is connected. 
■ The other class of resets is soft resets. As 
. explained above, soft resets clear the state engines 
and temporary registers in system 10 but they do 

70 not change configuration information, such as the 
mode bits in the cross-links. In addition, soft resets 
also clear the error handling mechanisms in the 
modules, but they do not change error registers 
such as system error register 898 and system fault 

75 address register 865. 

Soft resets are targeted so that only the neces- 
sary portions of the system are reset. For example, ' 
if module Interconnect 130 needs to be reset. CPU 
40 is not reset nor are the devices connected to I/O 

20 module 100. 

There are three unique aspects of soft resets. 
One is that each zone is responsible for generating 
its own reset. Faulty error or reset logic in one 
zone is thus prevented from causing resets in the 

25 non-faulted zone. 

The second aspect is that the soft reset does 
not disrupt the sequence of Instruction execution. 
CPUs 40, 40', 50, 50' are reset on a combined 
clock and hard reset only. Additionally memory 

30 controllers 70, 75, 70' and 75' have those state 
engines and registers necessary to service CPU 
Instructions attached to hard reset. Thus the soft 
reset is transparent to software execution. 

The third aspect is that the range of a soft 

35 reset, that is the number of elements in system 10 
that is affected by a soft reset is dependent upon 
the mode of system 10 and the original reset 
request. In full duplex mode, the soft reset request 
originating in CPU module 30 will issue a soft reset 

40 to all elements of CPU module 30 as well as all 
firewalls 1000 and 1010 attached to module inter- 
' connect 130 and 132. Thus all modules serviced 
by module interconnect 130 and 132 will have their 
state engines and temporary registers reset. This 

45 , will clear the system pipeline of any problem caus- 
ed by a transient error. Since system 10 is in 
duplex mode, zone 1l' will be doing everything 
that zone 11 is. Thus CPU module 30' will, at the 
same time as CPU module 30, issue a soft reset 

50 request. The soft reset in zone 1l' will have the 
same effect as the soft reset in zone 11. 

When system 10 is in a master/slave mode, 
however, with CPU module 30' in the slave mode, 
a soft reset request originating in CPU module 30 

55 will, as expected, issue a soft reset to all elements 
of CPU module 30 as well as all firewalls 1000 and 
1010 attached to module interconnects 130 and 
132. Additionally, the soft reset request will be 



22 



43 EP 0 415 551 A2 44 



forwarded to GPU module 30- via cross-links 90 
and 90', cross-link cables 25, and cross-links 90' 
and 95'. Parts of rnodule Interconnects 130' and 
132' will receive the soft reset. In this same con- 
figuration, a soft reset request originating from CPU 
module 30 will only reset memory controllers 70' 
and 75 and portions of cross-links 90' and 95'. 

Soft resets Include "CPU soft resets" and 
"system soft resets." A CPU soft reset Is a soft 
reset that affects the state engines on the CPU 
module that originated the request. A system soft 
reset is a soft reset over the module interconnect 
and those elements directly attached to It. A CPU 
module can always request a CPU soft reset. A 
system soft reset can only be requested If the 
cross-link of the requesting CPU is In duplex mode, 
master/slave mode, or off mode. A cross-link in the 
slave mode will take a system soft reset from the 
other zone and generate a system soft reset to its 
own module interconnects. 

CPU soft resets clear the CPU pipeline follow- 
ing an error condition. The CPU pipeline includes 
memory interconnects 80 and 82, latches (not 
shown) in memory controllers 70 and 75, DMA 
engine 800. and cross-links 90 and 95. The CPU 
soft' reset can also occur following a DMA or I/O 
time-out A DMA or I/O time-out occurs when the 
I/O device does not respond within a specified time 
period to a DMA or an I/O request 

Rg. 21 shows the reset lines from the CPU 
modules 30 and 30' to the I/O modules 1O0, 110, 
100 , and 110 and to the memory modules 60 and 
60'. The CPU module 30 receives a DC OK signal 
indicating when the power supply has settled. It is 
this signal which initializes the power-up reset. 
CPU module 30 receives a similar signal from its 
power supply. 

One system hard reset line is sent to each I/O 
module, and one system soft reset Is sent to every 
three I/O modules. The reason that single hard 
reset is needed for each module is because the 
system hard reset line are used to remove individ- 
ual I/O modules from system 10. The limitation of 
three I/O modules for each system soft reset is 
merely a loading consideration, in addition, one 
clock reset line is sent for every module and mem- 
ory module. The reason for using a single line per 
module is to control the skew by controlling the 
load. 

Rg. 22 shows the elements of CPU module 30 

which relate to resets. CPUs 40 and 50 contain 
clock generators 2210 and 2211. respectively. 
Memory controllers 70 and 75 contain clock gener- 
ators 2220 and 2221. respectively, and cross-links 
90 and 95 contain clock generators 2260 and 2261 , 
respectively. The clock generators divide down. the 
system clock signals for use by the individual mod- 
ules. 



Memory controller 70 contains reset control 
circuitry 2230 and a soft reset request register 
.2235. Memory controller 75 contains reset control 
circuitry 2231 and a soft reset . request register 
5 2236. 

Cross-link 90 contains both a local reset gener- 
ator 2240 and a system reset generator 2250. 
Cross-link 95 contains a local reset generator 2241 
and a system reset generator 2251. The "local" 

10 portion of a cross-link is that portion of the cross- 
link which remains with the CPU module when that 
cross-link is in the slave mode and therefore in- 
cludes the serial registers and some of the parallel 
registers. The "system" portion of a cross-link is 

75 that portion of the cross-link that is needed for 
access to module interconnects 130 and 132 (or 
130' and 132') and cross-link cables 25. 

The local reset generators 2240 and 2241 gen- 
erate resets for CPU module 30 by sending hard 

20 and soft reset signals to the local reset control 
circuits 2245 and 2246 of cross-links 90 and 95. 
respectively, and to the reset control circuits 2230 
and 2231 of memory controller 70 and 75, respec- 
tively. Local cross-link reset control circuits 2245 

26 and 2246 respond to the soft reset signals by 
resetting their state engines, the latches storing 
data to be transferred, and their error registers. 
Those circuits respond to the hard reset signals by 
taking the same actions as are taken for the soft 

30 resets, and by also resetting the error registers and 
the configuration registers. Reset control circuits 
2230 and 2231 respond to hard and soft reset 
signals In a similar manner. 

In addition, the local reset generator 2240 

35 sends clock reset signals to the I/O modules 100. 
110 and 120 via module interconnects 130 and 
132. The I/O modules 100. 110, and 120 use the 
clock reset signals to reset their clocks in the 
manner described below. Soft reset request regls- 

40 ters 2235 and 2236 send soft request signals to 
local reset generators 2240 and 2241. respectively. 

System reset generators 2250 and 2251 of 
cross-links 90 and 95, respectively, send system 
hard reset signals and system soft reset signals to 

45 I/O modules 100, 110. and 120 via module inter- 
connects 130 and 132, respectively. I/O modules 
100. 1 10, and 120 respond to the soft reset signals 
by resetting all registers that are dependent on 
CPU data or commands. Those modules respond 

50 to the hard reset signals by resetting the same 
register as soft resets do. and by also resetting any 
configuration registers. 

In addition, the system reset generators 2250 
and 2251 also send- the system soft and system 

55 hard reset signals to the system reset control cir- 
cuit 2255 and 2256 of each cross-link. System 
reset control circuit 2255 and 2256 respond to the 
system soft reset signals and to the system hard 
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reset signals in a manner similar to the response of 
the local reset control circuits to the local soft and 
local hard reset signals. 

Memory controllers 70 and 75 cause cross- 
links 90 and 95. respectively, to generate the soft 
resets when CPUs 40 and 50, respectively, write 
the appropriate codes Into soft reset request regis- 
ters 2235 and 2236, respectively. Soft reset re- 
quest registers 2235 and .2236 send soft reset 
request signals to local reset generators 2240 and 
2241, respectively. The coded error signal is sent 
from memory controller 70 to local reset generators 
2240 and 2241, 

System soft resets are sent between zones 
along the same data paths data and control signals 
are sent. Thus, the same philosophy of equalizing 
delays is used for resets as for data and ad- 
dresses, and resets reach all of the elements in 
both zones at approximately the same time. 

Hard resets are generated by CPUs 40 and 50 
writing the appropriate code into the local hard 
reset registers 2243 or by the request for a power 
up reset caused by the DC OK signal. 

Synchronization circuit 2270 in cross-link 90 
includes appropriate delay elements to ensure that 
the DC OK signal goes to all of the local and reset 
generators 2240, 2250, 2241 and 2251 at the same 
time. 

In fact, synchronization of resets is very Impor- 
tant In system 10. That is why the reset signals 
originate in the cross-links. In that way, the resets 
can be sent to arrive at different modules and 
elements in the modules approximately synchro- 
nously. 

With the understanding of the structure in Figs. 
21 and 22. the execution of the different hard 
resets can be better understood. The power up 
reset generates both a system hard reset, a local 
hard reset and a clock reset. Generally, cross-links 
90, 95. 90 and 95' are initially in both the cross- 
link off and resync off modes, and with both zones 
asserting clock mastership. 

The CPU/MEM fault reset is automatically ac- 
tivated whenever memory controllers 70, 75, 70' 
and 75' detect a CPU/MEM fault. The coded error 
logic is sent from error logic 2237 and 2238 to both 
cross-links 90 and 95. The CPU module which 
generated the fault is then removed from system 
10 by setting Its cross-link to the slave state and 
by setting the cross-link in the other CPU module 
to the master state. The non-faulting CPU module 
will not experience a reset, however. Instead, it will 
be notified of the fault in the other module through 
a code in a serial cross-link error register (not 
shown). The CPU/MEM fault reset consists of a 
clock reset to the zone with the failing CPU module 
and a local soft reset to that module. 

A resync reset is essentially a system soft 



reset with a local hard reset and a clock reset. The 
resync reset is used to bring two zones into lock- 
step synchronization. If, after a period In which 
zones 1 1 and 1 1 ' were not synchronized, the con- 

5 tents of the memory modules 60 and 60', including 
the stored states of the CPU registers, are set 
equal to each other, the resync reset is used to 
bring the zones into a compatible configuration so 
they can restart in a duplex mode. 

10 The. resync rieset is essentially a CPU hard 
reset and a clock reset. The resync reset is ac- 
tivated by software writing the resync reset address 
Into one of the parallel cross-link registers. At that 
time, one zone should be in the cross-link 

T6 master/resync master mode and the other in the 
cross-link slave/resync slave mode. A simultaneous 
reset will then be performed on both the zones 
which, among other things, will set all four cross- 
links into the duplex mode. Since the resync reset 

20 is not a system soft reset, the I/O modules do not 
receive reset. 

The preferred embodiment of system 10 also 
ensures that clock reset signals do not reset con- 
forming clocks, only non-conforming clocks. The 

25 reason for this is that whenever a clock is reset, it 
alters the timing of the clocks which in turn affects 
the operation of the modules with such clocks. -If 
the module was performing correctly and its clock 
was In the proper phase, then altering its operation 

30 would be both unnecessary and wasteful. 

Rg. 23 shows a preferred embodiment of cir- 
cuitry which will ensure that only nonconforming 
clocks are reset. The circuitry shown in Rg,.23 
preferably resides In the clock generators 2210, 

35 2211. 2220, 2221, 2260. . and 2261 of the cor- 
responding modules shown in Rg. 22. 

In the preferred embodiment, the different 
clock generators 2210. 2211, 2220. 2221. 2260, 
and 2261 include a rising edge detector 2300 and 

40 a phase generator 2310. The rising edge detector 
. 2300 receives the clock reset signals from the 
cross-links 90 and 95 and generates a pulse of 
known duration concurrent with the rising edge of 
the clock reset signal. That pulse is in an input to 

45 the phase generator 2310 as are the Internal clock 
signals for the particular module. The internal clock 
signals for that module are clock signals which are 
derived from the system clock signals that have 
been distributed from oscillator systems 200 and 

50 200'. 

Phase generator 2310 is preferably a divide- 
down circuit which forms different phases for the 
clock signals. Other designs for phase generator 
2310, such as recirculating shift registers, can also 
55 be used. 

Preferably, the rising edge pulse from rising 
edge detector 2300 causes phase generator 2310 
to output a preselected phase. Thus, for example, if 
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phase generator 2310 were a divide-down circuit 
with several stages, the clock reset rising edge 
pulse could be a set input to the stage which 
generates the preselected phase and a reset input 
to all other stages, if phase generator 2310 were 
already generating that phase, then the presence of 
the synchronized clock reset signal would be es- 
sentially transparent. 

The resets thus organized are designed to pro- 
vide the minimal dismption to the normal execution 
of system 10, and only cause the drastic action of 
interrupting the normal sequences of instruction 
execution when such drastic action Is required. 
This is particularly important in a dual or multiple 
zone environment because of the problems of re- 
synchronizatlon which conventional resets cause. 
Thus, it is preferable to minimize the number of 
hard resets, as is done in system 10. 



3. Protocol for Data Transfers 

The protocol for transferring data between the 
CPU modules 30 and 30', and the I/O modules 
100. lOO'. 110. 110\ 120, and 120'. will now be 
described. Data can be transferred' to or from a 
system resource, such as a disk 1075 or 1075', 
using I/O read and write cycles or using DMA. 
Typically, most data transfers with I/O devices are 
performed with DMA. However, In either situation, 
the data transfers occur between the memory con- 
trollers 70. 70'. 75. and 75' and the I/O modules 
100. lOO'. 110. IIO'. 120, and 120'. As a result, 
information is transmitted on dual rail module inter- 
connects 130, 13o', 132, and 132', which will be 
referred to as system resource buses for purposes 
of this discussion, as well as on- crosslink pathway 
25. 

In these data transfers, the presence of cros- 
slinks 90. 90', 95, and 95' coupling zone 11 to 
zone 1l' allows each CPU module 30 and 30' to 
communicate with any of the I/O modules in com- 
puter system 10. even if the module is In another 
zone. As a result, special circuitry and protocols 
are used in order to make use of the available 
interconnections during data transfers. 

For example, if zone 11 is In the master mode 
and zone 1l' is in the slave mode, then CPU 
module 30 in zone 11, which is the master, can 
transfer data to and from I/O modules' In its zone 
1 1 and in the other zone 1 1'. In master/slave mode, 
when information is transmitted from any one of the 
I/O modules In either zone, both of the memory 
controllers 70 and 75 in the master zone 1 1 receive 
the same Information. When Information Is sent 
from memory controllers 70 and 75 in the master 
zone 1 1 to one of the I/O modules in either zone, 
the same information is received at both of the 



firewalls 1000 and 1010 in that I/O module. 

In full duplex mode, both zones operate to- 
gether in lockstep synchronization and achieve a 
fault tolerant computer system. Both of the CPU 

5 modules 30 and 30 receive the same Information in 
full duplex mode when infonmatlon is transmitted 
from any I/O module in either zone. Furthermore, in 
full duplex mode, when Information is sent to an I/O 
module in either zone, the same Information is 

70 received by the I/O module at firewalls 1000 and 
1010 from CPU modules 30 and 30'. 

During data transfers, the crosslinks 90, 90', 
95, and 95 are used as data routers. The switching 
logic in the crosslinks shown in Figures 11 and 12 

75 performs the function of coupling one pair of mod- 
ule Interconnects 130 and 132, or 130' and 132*. to 
specific memory controllers. The path that is cre- 
ated using this switching logic may depend on the' 
. direction of the data transfer, on whether the 

20 source or destination for the information is an I/O 
module, on which module Interconnect buses will 
be involved In the data transfer, and on the mode 
of operation of the crosslinks. 

As shown In Figure 3, a pair of parallel buses 

25 88 and 89 couples memory controllers 70 and 75 
to crosslinks 90 and 95. respectively. 
Crosslink/memory controller buses 88 and 89 carry 
read, write, or DMA data and read or write ad- 
dresses on sixteen bidirectional data lines, and 

30 contain additional unidirectional control lines, both 
up (CSUP-to the memory controller) and down 
(CSDN-from the memory controller), for carrying 
• control codes. Similarly, the system resource bus- 
es (i.e.. module interconnects 130 and 132) also 

35 contain sixteen bidirectional data lines for carrying 
read or write addresses and read, write, or DMA 
data, and additional unidirectional control lines for 
carrying control codes. 

Transfers of data via DMA will be described 

40 first. 

Prior to transmission of DMA data between a 
system resource, e.g.. I/O module 100 coupled to 
disk 1075, and a CPU module, e.g., CPU module 
30, the switching logic in the crosslinks 90. 90', 95, 

45 and 95' must be configured property so that control 
codes and DMA data will be sent to the appropriate 
components of the computer system. Therefore, 
DMA setup information Is transmitted by CPU mod- 
ule 30 from memory controllers 70 and 75. 

50 DMA setup information is transmitted from 
memory controllers 70 and 75 to crosslinks 90 and 
95 via the pair of crosslink/memory controller bus- 
es 88 and 89. This DMA setup Information contains 
bits, in either the control code, the write address, 

55 and/or the write data, which are used by the com- 
ponents in the computer system, including the 
crosslinks, to uniquely designate a setup write 
transaction (cycle). In the preferred embodiment of 

25 
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the . invention, the setup write transaction is iden- 
tified by a control code transnnitted on the unidirec- 
tional control lines of crosslink/nnemory controller 
buses 88 and 89 down from CPU module 30 to 
crosslinks 90 and 95. These lines are refenred to as 5 
CSDN (control and status down) lines. 

A setup write transaction Is used to predefine 
the path (between a memory controller and a sys- 
tem resource) that must be set up for a subsequent 
DMA transfer. During write transactions, the data io 
provided by CPU module 30 on the sixteen bidirec- 
tional data lines of crosslink/memory controller bus- 
es 88 and 89 consists of a write address followed 
by write data. As embodied herein, a setup write 
transaction has the same format as other types of 75 
write transactions, except the control code on the 
CSDN lines during setup write transactions is dif- 
ferent from the control codes used for other write 
transactions. 

The DMA setup information from the memory 20 
controller may be included in either the control 
code, the write address, and/or the write data. In 
the preferred embodiment of the invention, one bit 
in the write address Indicates which zone the DMA 
transfer will involve. By monitoring this bit In the 26 
write address, crosslinks 90 and 95 determine 
which pair of system resource buses will be in- 
volved in the DMA transfer. The write address also 
is monitored by4he crosslinks and compared with 
the locations In the I/O space of computer system 30 
10 to determine whether the DMA transfer will 
involve an I/O module. Furthermore, the write data 
transmitted during the setup write transaction in- 
cludes one or more bits for indicating the direction 
•of the DMA transfer. 35 

Therefore, in the preferred embodiment of the 
invention, crosslinks 90 and 95 monitor the control 
code, the write address, and the write data to 
decide whether DMA setup information is being 
transmitted, to determine the DMA transfer direc- 4o 
tion. to identify the pair of system resource buses 
(module Interconnects) that will be involved, and to 
ascertain whether the DMA transfer will involve a 
system resource (I/O module) coupled to any of 
the system resource buses. Furthermore, as shown 4S 
In Figure 10, crosslink 90 contains crosslink mode 
bits 916 in control and status register 912 that 
identify the state of the crosslink, i.e., either the 
crosslink is off, is a master, is a slave, or is 
operating In duplex mode. This Infonmation Is used so 
in routing control codes and DMA data during a 
DMA transfer. 

In accordance with the present invention, each 
of the crosslinks includes means for decoding DMA 
setup information during a setup write transaction. 55 
The switching logic in the crosslinks performs the 
function of data routing under the control of a 
number of decoders. When a setup write transac- 



551 A2 




tion is initiated by a memory controller, these de- 
coders determine the direction of an upcoming 
DMA transfer, determine which pair of system re- 
source buses the DMA transfer will involve, and 
determine whether the DMA transfer will Involve a 
system resource coupled to one pair of the system 
resource buses. 

Decoders 971, 996, and 998. which are shown 
in Figures 11 and 12, decode the DMA setup 
Information transmitted during a setup write trans- 
action to generate the internal control signals re- 
quired to set up paths between memory controllers 
and one pair of system resource buses. Rgure 11 
shows the switching logic Involved in the transfer of 
control codes between the unidirectional control 
lines, CSUP and CSDN, of crosslink/memory con- 
troller buses 88 and 89 and of module intercon- 
nects 130 and 132. Figure 12 shows the switching 
logic involved in the transfer of data between the 
bidirectional data lines of crosslink/memory control- 
ler buses 88 and 89 and of module interconnects 
130 and 132, 

Preferably, the CPU module and the system 
resource include pointer register means for indicat- 
ing a DMA data address to be accessed by the 
CPU module and' a DMA data address to be acces- 
sed by the system resource. As shown in Figure 8, 
In memory controller 70 there are DMA registers 
830 in DMA engine 800. DMA registers 830 include 
DMA address register 836, which is set to an initial 
value by DMA engine 800, In the I/O modules, the 
address in local RAM 1060 that will be accessed 
during a DMA operation is provided by a DMA 
address register in shared memory controller 1050, 
which Is shown In Figure 17. 

In the preferred embodiment, a write transac- 
tion Is Initiated by CPU module 30, and memory 
controller 70 transmits DMA starting address in- 
formation during this write transaction. The DMA 
starting address infonmation specifies a DMA data 
address to be accessed by a system resource 
during an upcoming DMA transfer, and, this in- 
formation is transmitted to a system resource from 
memory controller 70 via the crosslink/memory 
controller bus 88. via crosslink 90. and (if the 
system resource is in zone 11) via module inter- 
connect 130, or (if the system resource is in zone 
11') via crosslink pathway 25, crosslink 90', and 
module interconnect 130'. The DMA data address 
is then stored in the pointer register, in a system 
resource, such as in shared memory controller 
1050 in I/O module 100. 

Similariy, DMA engine 800 specifies a DMA 
data address in memory 60 to be accessed by 
memory controller 70 during an upcoming DMA 
transfer. This DMA starting address information is 
written into the pointer register in the memory 
controller, I.e., DMA address register 836. 
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As embodied herein, the CPU module and the 
system resource Include count register means for 
indicating the number of bytes of DMA data to be 
accessed by the CPU module and by the system 
resource during an upcoming DMA transfer. As 
shown in Rgure 8, DMA registers 830 in DMA 
engine 800 include DMA byte counter register 832, 
which is set to an initial value by DMA engine 800 
during set-up. In the 1/0 modules. DMA control 
1890 in firewall 1000. which Is shown In Figure 18. 
keeps track of how many words or bytes are left to 
be transferred via DMA. 

Preferably, a write transaction is Initiated by 
CPU module 30. and memory controller 70 trans- 
mits DMA count information during this write trans- 
action. The DMA count information specifies a 
DMA count of the number of bytes (words) of DMA 
data to be accessed during an upcoming DMA 
transfer, and this information is transmitted to a 
system resource from memory controller 70 via the 
crosslink/memory controller bus 88. via crosslink 
90, and (if the system resource is in zone 11) via 
module interconnect 130, or (if the system re- 
source Is in zone 11) via crosslink pathway 25, 
crosslink 90', and module interconnect 130'. The 
DMA count is then stored in the count register In a 
system resource, such as in DMA control 1890 in 
firewall 1000 in I/O module 100. At the same time, 
this DMA count also is written Into the pointer 
register in the memory controller, i.e., DMA byte 
counter register 832. 

In the prefen-ed embodiment of the Invention, 
the DMA count is transmitted from memory control- 
ler 70 during a setup write transaction. The write 
data transmitted to a system resource during the 
setup write transaction includes DMA count in- 
formation. As embodied herein, one bit in the write 
address transmitted during the setup write transac- 
tion identifies the zone (and therefore the pair of 
module interconnects) that will be involved in the 
DMA transfer, and if the write address corresponds 
to an address in the I/O space of system 10, a 
system resource coupled to one pair of the module 
interconnects will be involved in the DMA transfer. 
Finally, the write data transmitted during the setup 
write transaction preferably Includes bits specifying 
the direction of the DMA transfer. 

Reference should be made to Figures 24A-E, 
which are timing diagrams illustrating the transfer 
of control codes and DMA data on module inter- 
connects 130 and 132 (Figures 24A, B, C, & E) or 
on crosslink/memory controller buses 88 and 89 
(Figure 24D) during a DMA transfer. 

In accordance with the invention, a start DMA 
code is transmitted from memory controllers 70 
and 75 to cross links 90 and 95 on the unidirec- 
tional control lines CSDN of buses 88 and 89. As 
shown In Rgures 24A-24E. during each attempted 
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DMA transfer, the first control code transmitted is a 
START code on the CSDN lines. The start DMA 
code is transmitted to one of the system resources 
via the crosslinks and via the pair of system re- 

5 source buses coupled to the system resource. 

The switching logic in crosslinks 90. 90', 95, 
and 95' Is configured in accordance with the de- 
coded DMA setup information in response to trans- 
■ mission of the DMA start code. The switching logic 

70 is not triggered until the DMA start code Is re- 
ceived. 

In crosslink. 90 in the following examples, dri- 
vers 940, 946. 951 , 969, and 992 are activated and 
deactivated by internal control signals from de- 

75 coder 998. Decoder 998 also generates Internal 
control signals that select which one of two inputs 
are routed through multiplexers CSMUXD 945. 
MUXC 976, and MUXD 982. Drivers 937 and 984 
are activated and deactivated by decoders 971 and 

20 996, so that control codes and DMA data from 
memory controller 70 can be transmitted to cros- 
slink 90' in zone I1'. 

In one example, the switching logic is config- 
ured to forward DMA data from the pair of system 

25 resource buses 130 and '132, via the switching 
logic in the pair of crosslinks 90 and 95 to the pair 
of memory controllers 70 and 75, and via the 
switching logic in the pair of crosslinks 90 and 95 
and the switching logic In the pair of crosslinks 90 

30 and 95 to the pair of memory controllers 70 and 
75'. This path is set up at times when the DMA 
setup information decoded by decoders 971. 996, 
and 998 Indicates that the DMA transfer is in the 
up direction, will involve zone 11 (and therefore the 

35 pair of module interconnects 130 and 132), and will 
involve a system resource (e.g., I/O module 100) 
coupled to one pair of module interconnects. 

In this example, driver 969 is deactivated and 
driver 992 is activated, provided crosslink mode 

40 bits 916 indicate crosslink 90 is a master or is 
operating In duplex mode, because .the DMA setup 
information indicates the DMA transfer is in the up 
direction, MUXC 976 selects the input from module 
interconnect 130 because the DMA setup informa- 

45 tlon Indicates that the transfer is in the up direction. 
MUXD 982 selects the input from module intercon- 
nect 130 because the DMA setup information in- 
dicates that the transfer involves zone 11. As a 
result, DMA data is transmitted from module inter- 

50 connect 130 to memory controller 70. 

Driver 951 is activated during this example if 
crosslink mode bits 916 indicate crosslink 90 Is a 
master or is operating in duplex mode. CSMUXD 
945 selects the input from module interconnect 130 

55 because the DMA setup Infomnatlon Indicates that 
the transfer will involve zone 11. As a result, control 
codes are transmitted from module interconnect 
1 30 to memory controller 70. 
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In this example, drivers 946 and 984 are ac- 
tivated, provided crosslink 90 is a master or is 
operating in duplex mode, because the DMA setup 
information indicates that the transfer involves an 
I/O module in zone 11. As a result; control codes 
and DMA data from, module interconnect 130 will 
be transmitted from crosslink 90 to memory con- 
troller 70' via crosslink 90'. 

Driver 940 is activated and driver 937 is deacti- 
vated In this first example because the DMA setup 
information indicates that the transfer involves an 
I/O module in zone 1 1 . Control codes are transmit- 
ted to module Interconnect 130, either from mem- 
ory controller 70, or from memory controller 7o' via 
crosslink 90' and crosslink pathway 25. When cros- 
slink mode bits 916 indicate crosslink 90 is a 
master, CSMUXA 935 selects the input from mem- 
ory controller 70. When crosslink mode bits 916 
indicate crosslink 90 is a slave, CSMUXA 935 
selects the input from crosslink pathway 25. When 
the crosslinks are operating In duplex mode, either 
CSMUXA 935 selects the input from memory con- 
troller 70 while CSMUXA 935m selects the Input 
from crosslink pathway 25, or CSMUXA 935 se- 
lects the input from crosslink pathway 25 while 
CSMUXA 935m selects the input from memory 
controller 75. As a result, in duplex mode, the 
control codes transmitted to module interconnects 
130 and 132 and received at firewalls 1000 and 
1010 of I/O module 100 are from two different 
zones. 

In a second example, the switching logic is 
configured to fon^^ard DMA data from the pair of 
system resource buses 130' and 132', via the 
switching logic in the pair of crosslinks 90' and 95' 
to the pair or memory controllers 70' and 75', and 
via the switching logic In the pair of crosslinks 90' 
and 95 to the switching logic in the pair of cros- 
slinks 90 and 95 and to the pair of memory control- 
lers 70 and 75. This path is set up at times when 
the DMA setup Information decoded by decoders 
971, 996, and 998 Indicates that the DMA transfer 
is in the -up direction, will involve zone 11 ' (and 
therefore the pair of module Interconnects 130' and 
132'), and will involve a system resource (e.g., I/O 
module 100 ) coupled to one pair of module inter- 
connects. 

In this second example, driver 969 is deacti- 
vated and driver 992 is activated, provided cros- 
slink mode bits 916 indicate crosslink 90 is a 
master or is operating in duplex mode, because the 
DMA setup Infomnation indicates the DMA transfer 
is in the up direction. MUXD 982 selects the input 
from crosslink pathway 25 because the DMA setup 
information Indicates that the transfer Involves zone 
1l'. As a result, DMA data is transmitted from 
module interconnect 1 30' to memory controller 70 
via crosslink 90' and crosslink pathway 25. 
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Driver 951 is activated during this example if 
crosslink mode bits 916 indicate crosslink 90 Is a 
master or is operating in duplex mode. CSMUXD 
945 selects the input from crosslink patiiway 25 
6 because tiie DMA setup information indicates that 
the transfer will involve zone 1l'. As a result, 
control codes are transmitted from module inter- ■ 
connect 13o' to memory controller 70 via crosslink 
90' and crosslink pathway 25. - 

10 In this example, driver 946 is deactivated be- 
cause the DMA setup information indicates tiiat the 
transfer involves zone 1l'. Driver 984 is deacti- 
vated because the DMA setup Information Indicates 
that the transfer is in the up direction and involves 

76 zone 1l'. However, control codes and DMA data 
from module interconnect 130' will be transmitted 
from crosslink 90' to memory controller 70 via 
receivers 947 and 986 in crosslink 90. 

Driver 940 is deactivated and driver 937 is 

20 activated in this second example because the DMA 
setup Information indicates that the transfer in- 
volves zone 11 . The switching logic in crosslink 
90' is configured to transmit control codes to mod- 
ule interconnect 1.30', either from memory control-. 

25 ler 70', or from memory controller 70 via crosslink 
90 and crosslink pathway 25. When crosslink mode 
bits 916 indicate crosslink 90 is a master, control 
codes are transmitted to module interconnect 130' 
from memory controller 70. When crosslink mode 

30 bits 916 indicate crosslink 90 is a slave, control 
codes are transmitted to module Interconnect 130' 
from memory controller 70', When the crosslinks 
are operating in duplex mode, either the switching 
. logic in crosslink 90' selects the input from mem- 
os ory controller 70 while the switching logic in cros- 
slink 95 selects the input from memory controller 
75 , or the switching logic in crosslink 90' selects 
the input from memory controller 70' while the 
switching logic in crosslink 95' selects tiie input 

40 from memory controller 75. As a result, in duplex 
mode, the control codes transmitted to module 
interconnects 130' and 132' and received at 
firewalls 1000 and 1010 of I/O module lOO' are 
from two different zones. 

45 In a third example, tiie switching logic Is con- 
figured to forward DMA data, from either memory 
controller 70 or 75 via the switching logic in cros- 
slink 90 or 95. and from eitiier memory controller 
70' or 75' via the switching logic in crosslinks 90' 

50 and 90 or crosslinks 95' and 95, to the pair of 
system resource buses 130 and 132. This path is 
set up at times when the DMA setup information 
decoded by decoders 971 , 996, and 998 indicates 
that ihe DMA transfer is in the down direction, will 

65 involve zone 11 (and therefore the pair, of module 
interconnects 130 and 132), and will Involve a 
system resource (e.g., I/O module 100) coupled to 
one pair of module Interconnects. 
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In this third example, driver 969 is activated 
and drivers 984 and 992 are deactivated because 
the DMA setup information indicates the DMA 
transfer is in the down direction arid will Involve an 
I/O module in zone 11. DMA data is transmitted to 
module Interconnect 130. either from memory con- 
troller 70, or from memory controller 70' via cros- 
slink 90 and crosslink pathway 25. When crosslink 
mode bits 916 indicate crosslink 90 Is a master. 
MUXA 966 selects the input from memory control- 
ler 70. When crosslink mode bits 916 indicate 
crosslink 90 is a slave, MUXA 966 selects the input 
from cross link pathway 25, When the cross links 
are operating in duplex mode, either MUXA 966 
selects the input from memory controller 70 while 
MUXA (not shown) in crosslink 95 selects the input 
from crosslink pathway 25, or MUXA 966 selects 
the Input from crosslink pathway 25 while MUXA in 
crosslink 95 selects the input from memory control- 
ler 75. As a result, in duplex mode, the DMA data 
transmitted to module interconnects 130 and 132 
and received at firewalls 1000 and 1010 of I/O 
module 1 00 is from two different zones. 

Driver 951 is activated during this example if 
crosslink mode bits 916 indicate crosslink 90 Is a 
master or is operating in duplex mode. CSMUXD 
945 selects the input from module interconnect 130 
because the DMA setup information indicates that 
the transfer will involve zone 11. As a result, control 
codes are transmitted from module interconnect 
1 30 to memory controller 70. 

In this example, driver 946 is activated, pro- 
vided crosslink 90 is a slave or is operating in 
duplex mode, because the DMA setup information 
Indicates that the transfer involves an I/O module in 
zone 11. As a result, control codes from module 
interconnect 130 will be transmitted from crosslink 
90 to memory controller 70 via crosslink 90*. 

Driver 940 is activated and driver 937 is deacti- 
vated in this third example because the DMA setup 
information indicates that the transfer involves an 
I/O module in zone 11. Control codes are transmit- 
ted to module interconnect 130, either from mem- 
ory controller 70, or from memory controller 70 via 
crosslink 90' and crosslink pathway 25. When cros- 
slink mode bits 916 indicate crosslink 90 is a 
master,* CSMUXA 935 selects the input from mem- 
ory controller 70. When crosslink mode bits 916 
indicate crosslink 90 is a slave, CSMUXA 935 
selects the inptit from crosslink pathway 25. When 
the crosslinks are operating in duplex mode, either 
CSMUXA 935 selects the input from memory con- 
troller 70 while CSMUXA 935m selects the input 
from crosslink pathway 25, or CSMUXA 935 se- 
lects the input from crosslink pathway 25 while 
CSMUXA 935m selects the input from memory 
controller 75. As a result, in duplex mode, the 
control codes transmitted to module Interconnects 
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130 and 132 and received at firewalls 1000 and 
1010 of I/O module 100 are from two different 
zones. 

In a fourth example, the switching logic is con- 

5 figured to forward DMA data, from either memory 
controller 70 or 75 via the switching logic in cros- 
slinks 90 and 90' or 95 and 95', and from either 
memory controller 70' or 75' via the switching logic 
in crosslinks 90' or 95 . to the pair of system 

10 resource buses 130' and 132'. This path is set up 
at times when the DMA setup information decoded 
by decoders 971, 996, and 998 indicates that the 
DMA transfer is in the down direction, will involve 
zone 1l' (and therefore the pair of module inter- 

75 connects 130 and 132 ), and will involve a system 
resource (e.g., I/O module lOO') coupled to one 
pair of module interconnects. 

In this fourth example, driver 984 is activated, 
drivers 969 and 992 are deactivated, and MUXC 

20 976 selects the input from memory controller 70 . 
because the DMA setup information indicates the 
DMA transfer is in the down direction and will 
involve zone 1 1 . DMA data is transmitted to mod- 
ule interconnect 130' via crosslink 90'. either from 

25 memory controller 70 , or from memory controller 
70 via crosslink 90 and crosslink pathway 25. 
When crosslink mode bits 916 indicate crosslink 90 
is a master, the switching logic in crosslink 90' 
selects the input from memory controller 70 . When 

30 crosslink mode bits 916 indicate crosslink 90 is a. 
slave, the switching logic in crosslink 90' selects 
the Input from crosslink pathway 25, When the 
crosslinks are operating in duplex mode, either the 
switching logic in crosslink 90' selects the input 

35 from memory controller 70' while the switching 
logic in crosslink 95' selects the input from cros- 
slink pathway 25. or the switching logic in crosslink 
90 selects the input from crosslink pathway 25 
while the switching logic in crosslink 95 selects the 

40 input from memory controller 75 . As a result, in 
duplex mode, the DMA data transmitted to module 
interconnects 130' and 132' and received at 
firewalls 1000 and 1010 of I/O module lOO' is from 
two different zones. 

45 Driver 951 is activated during this example if 
crosslink mode bits 916 indicate crosslink 90 is a 
master or" Is operating in duplex mode. CSMUXD 
945 selects the input from crosslink pathway 25 
because the DMA setup information indicates that 

50 the transfer will involve zone 1l'. As a result, 
control codes are transmitted from module inter- 
connect 130' to memory controller 70. 

In this example, driver 946 is deactivated be- 
cause the DMA setup information Indicates that the 

55 transfer involves zone 1 1 . However, control codes 
from module interconnect 130 will be transmitted 
from crosslink 90' to memory controller 70 via 
receiver 947 in crosslink 90. 
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Driver 937 is activated and driver 940 is deacti- 
vated in this fourth example because the DMA 
setup infornnation indicates that the transfer in- 
volves zone 1 1 Control codes are transmitted to 
module interconnect 130' via crosslink 90', either s 
from memory controller 70', or from memory con- 
troller 70 via crosslink 90 and crosslink pathway 25. 
When crosslink mode bits 916 indicate crosslink 90 
is a slave, the switching logic in crosslink 90' 
selects the input from memory controller 70'. When io 
crosslink mode bits 916 indicate crosslink 90 is a 
master, the switching logic in crosslink 90' selects 
the input from crosslink pathway 25. When the 
crosslinks are operating in duplex mode, either the 
switching logic in crosslink 90' selects the input 75 
from memory controller 70' while the switching 
logic in crosslink 95 selects the input from cros- 
slink pathway 25. or the switching logic in crosslink 
90 selects the input from crosslink pathway 25 
while the switching logic in crosslink 95' selects the 20 
input from memory controller 75'. As a result, in 
duplex mode, the control codes transmitted to 
module interconnects 130' and 132' and received 
at firewalls 1000 and 1010 of 1/0 module IOO' are 
from two different zones. 25 

In accordance with the invention, in response 
to transmission of a start DMA code to one of the 
system resources (e.g., I/O module 100), an ac- 
knowledge code is transmitted to the memory con- 
trollers. As shown in Rgures 24A-24E, during each 30 
attempted DMA transfer, the first control code 
transmitted on the CSUP lines is an ACK code. 
Preferably, a system resource transmits an ac- 
knowledge code to the memory controllers In re- 
sponse to a start DMA code to indicate that the 35 
system resource has control of the pair of system 
resource buses coupled to the system resource. 

In this example, ACR codes are transmitted by 
firewalls 1000 and 1010 in I/O module 100 to the 
crosslinks 90 and 95 via the pair of module inter- 40 
connects 130 and 132 to indicate that firewalls 
1000 and 1010 have control of II32T bus 1020. 
Memory controllers 70 and 75 receive ACK codes 
from crosslinks 90 and 95 on the unidirectional 
control lines CSUP of buses 88 and 89. The ACK 45 
codes also are transmitted from crosslinks 90 and 
95 to memory controllers 70' and 75' via crosslinks 
90 and 95' and via buses BS' and 89'. 

In the present invention, at times when the 
DMA transfer is in the up direction, and after trans- 50 
mission of an acknowledge code from a system 
resource, the system resource transmits DMA data 
to the memory controllers, via the pair of system 
resource buses coupled to the system resource, 
and via the switching logic in the crosslinks. Prefer- 55 
ably, as shown in Rgures 24C-24E, during DMA 
transfers from a system resource (e.g., I/O module 
100) to the memory controllers, a DMA code Is 
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transmitted on the CS|UP lines of module Intercon- 
nects 130 and 132 to crosslinks 90 and 95. The 
DMA code then is transmitted from crosslinks 90 
and 95, to the CSUP lines of crosslink/memory 
controller buses 88 and 89 and (via crosslinks 90' 
and 95') to the CSUP lines of crosslink/memory 
controller buses 88' and 89'. Transmission of the 
DMA code on the CSUP lines indicates data Is 
being sent to the memory controllers on the 
bidirectional data lines of module interconnects 130 
and 132 and crosslink/memory controller buses 88, 

89, 88'. and 89' 

DMA data is accessed by the system resource 
from the DMA data address Indicated by a pointer 
register in the system resource, e.g., by shared 
memory controller 1050 in I/O module 100. I/O 
module 100 drives the bidirectional data lines of 
module interconnects 130 and 132 with this DMA 
data, and crosslinks 90, 95, 90', and 95' drive the 
bidirectional data lines of buses 88, 89, 88 , and 
89' and fonfvard this DMA data to the memory 
controllers, as shown in Rgures 24C-E. 

In accordance with the invention, at times when 
the DMA transfer is in the down direction, and after 
transmission of an acknowledge code from a sys- 
tem resource, the memory controllers transmit 
DMA data to one of the system resources, via the 
switching logic in the crosslinks, and via the pair of 
system resource buses coupled to the system re- 
source. In the preferred embodiment shown in Rg- 
ures 24A-24B. during DMA transfers to a system 
resource (e.g., I/O module 100), a DMA code Is 
transmitted on the CSDN lines of crosslink/memory 
controller buses 88, 89, 88', and 89' to crosslinks 

90, 95. 90', and 95'. The DMA code then is trans- 
mitted from crosslinks 90 and 95 to the CSDN lines 
of module interconnects 130 and 132. Transmis- 
sion of the DMA code on the CSDN lines indicates 
data is being sent to the system resource on the 
bidirectional data lines of crosslink/memory control- 
ler buses 88, 89. 88'. and 89' and module intercon- 
nects 130 and 132. 

DMA data is accessed by each memory con- 
troller from the DMA data address in jmemory 60 or 
60 indicated by a pointer register in the memory 
controller, e.g., by DMA address register 836 In 
memory controller 70. Memory controllers 70, 75, 
70', and 75' drive the bidirectional data lines of 
buses. 88, 89, 88', and 89' with this DMA data, and 
crosslinks 90 and 95 drive the bidirectional data 
lines of module interconnects 130 and 132 and 
forward this DMA data to firewalls 1000 and 1010 
of I/O module 100, as shown in Rgures 24A-B. 

In a preferred embodiment of the invention, 
after transmission of an acknowledge code from a 
system resource (e.g.. I/O module 100), a stall 
code may be transmitted to the memory control- 
lers. As shown in Rgures 24A-24E, during DMA 
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transfers, a STALL code may be transmitted on the 
CSUP lines, ' Preferably, a system resource trans- 
mits a stall code to the memory controllers at times 
when the DMA transfer is in the down direction to 
indicate that the system resource is not ready to 
accept DMA data from the memory controllers. 
Similarly, the system resource transmits a stall 
code to the memory controllers at times when the 
DMA transfer is in the up direction to indicate that 
the system resource is not ready to send DMA 
data to the memory controllers. 

In this example, STALL codes are transmitted 
by firewalls 1000 and 1010 In I/O module 100 to 
the crosslinks 90 and 95 via the pair of module 
interconnects 130 and 132. Memory controllers 70 
and 75 receive STALL codes from crosslinks 90 
and 95 on the unidirectional control lines CSUP of 
buses 88 and 89. The STALL codes also are trans- 
mitted from crosslinks 90 and 95 to memory con- 
trollers 70' and 75' via crosslinks 90' and 95' and 
via buses 88' and 89'. 

In Rgures 24A and 24B, the transmission of 
control codes and DMA data on the module inter- 
connects is shown. The same control codes and 
DMA data also are transmitted on the 
crosslink/memory controller buses, but the period 
from transmission of a control code by a system 
resource on the CSUP lines until transmission of a 
response by the memory controllers on the CSON 
lines and the data lines is shorter on the 
crosslink/memory controller buses. 

As shown In Figures 24A and 24B, there is a 
delay before DMA data is transmitted on module 
interconnects 1 30 and 1 32 In response to transmis- 
sion of an ACK code, and there also is a delay 
before the memory controllers stop transmitting 
DMA data on module interconnects 130 and 132 in 
response to transmission of a STALL code by the 
system resource. The response times are longer 
for transfers of control codes and DMA data on the 
system resource buses because of the additional 
delay time introduced in order to route control 
codes and DMA data via crosslink pathway 25 and 
the switching logic in crosslinks 90, 95. 90', and 
95'. As a result, if a system resource transmits a 
STALL code indicating that it is not ready to accept 
DMA data from the memory controllers, the system 
resource still must have enough available storage 
to accept the additional DMA data that will be 
transmitted to the system resource before the 
memory controllers receive the STALL code. 

In accordance with the invention, after trans- 
mission of DMA data between the memory control- 
lers and one of the system resources (e.g., I/O 
module 100). a done code is transmitted to the 
memory controllers. Preferably, a system resource 
transmits a done code to the memory controllers to 
Indicate successful DMA data transmission. As 



shown in Figures 24A and 240, after a successful 
transmission of DMA data, a DONE code is trans- 
mitted on the CSUP lines. 

. As embodied herein, a DMA count transmitted 

5 to the firewalls of I/O module 100 during a setup 
write transaction indicates the number of bytes of 
DMA data to be accessed during the entire DMA 
transfer. Preferably, a number of subtransfers of 
DMA data are performed in order to transmit the 

10 number of bytes specified by the DMA count. A 
START code Is transmitted to begin each subtrans- 
fer, and a DONE code is transmitted to indicate 
completion of the subtransfer. 

In this example, DONE codes are transmitted 

76 by firewalls 1000 and 1010 in I/O module 100 to 
the crosslinks 90 and 95 via the pair of module 
interconnects 130 and 132 to indicate that firewalls 
1000 and 1010 have not detected a miscompare 
error during the preceding subtransfer of DMA 

20 data. At times when the DMA transfer is in the up 
direction, the transmission of DONE codes after the 
last subtransfer, i.e., after the number of bytes 
specified by the DMA count has been transmitted, 
also indicates that firewalls 1000 and 1010 have not 

25 detected a EDC/CRC error during the entire trans- 
fer of DMA data. Memory controllers 70 and 75 
receive DONE codes from crosslinks 90 and 95 on 
the CSUP lines of buses 88 and 89. The DONE 
codes also are transnnitted from crosslinks 90 and 

30 95 to memory controllers 70 and 75 via crosslinks 
90' and 95' and via buses 88' and 89'. 

In the preferred embodiment of the invention, 
after transmission of the acknowledge code from 
one of the system resources (e.g., I/O module 

35 100), an error code may be transmitted to the 
memory controllers. Preferably, a system resource 
transmits an error code to the memory controllers 
to indicate unsuccessful DMA data transmission. 
As shown in Figures 24B, 24D, and 24E, when an 

40 attempted transmission of DMA data is unsuccess- 
ful, an ERROR code or a CRC ERROR code is 
transmitted on the CSUP lines. 

In the example shown in Figure 248, at times 
when the DMA transfer is in the down direction, 

45 firewall comparison circuit 1840 (Rg. 18) is used 
by firewalls 1000 and 1010 in I/O module 100 to 
detect whether there is a miscompare error be- 
tween the two sets of DNA data and control codes 
transmitted to I/O module 100 on module intercon- 

50 nects 130 and 132. If firewalls 1000 and 1010 
detect a miscompare error during the subtransfer 
of control codes and DMA data. ERROR codes are 
transmitted by firewalls 1000 and 1010 in 1/0 mod- 
ule 100 to the crosslinks 90 and 95 via the pair of 

55 module interconnects 130 and 132. Memory con- 
trollers 70 and 75 receive ERROR codes from 
crosslinks 90 and 95 on the CSUP lines of buses 
88 and 89. The ERROR codes also are transmitted 
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from crosslinks 90 and 95 to memory controllers 
70 and 75' via crosslinks 90' and 95' and via 
buses 88' and 89'. 

In the example shown in Figure 24D, at times 
when the DNA transfer is in the up direction, logic 
gates 960 and 960m are used by crosslinks 90 and 
95 to detect whether there is a miscompare error 
between the two sets of control codes transmitted 
from I/O module 100 on module interconnects 130 
and 132. If logic gates 960 and 960m detect a 
miscompare error during the preceding subtransfer 
of control codes, ERROR codes are transmitted by 
crosslinks 90 and 95 to memory controllers 70 and 
75 on the CSUP lines of buses 88 and 89. Cros- 
slinks 90 and 95' also include logic gates for 
detecting whether there is a miscompare error be- 
tween two sets of control codes transmitted from 
I/O module 100 to crosslinks 9o' and 95' via the 
crosslink pathway, and transmit ERROR codes to 
memory controllers 70' and 75'. 

Finally, in the example shown in Figure 24E, at 
times when the DMA transfer is in the up direction, 
DNA control 1890 In firewalls 1000 and 1010 in I/O 
module 100 reads the generated EDC/CRC from 
EDG/CRC generator 1850 to detect whether there 
are any EDC/CRC en-ors in the DNA data transmit- 
ted by 1/0 module 100 to the memory controllers. 
This occurs only after the last subtransfer, i.e.. after 
the number of bytes specified by the DNA count 
has been transmitted. If firewalls 1000 and 1010 
detect a CRC error for the preceding transfers of 
DMA data, CRC ERROR codes are transmitted by 
firewalls 1000 and 1010 in I/O module 100 to the 
crosslinks 90 and 95 via the pair of module inter- 
connects 130 and 132. Memory controllers 70 and 
75 receive the CRC ERROR codes from crosslinks 
90 and 95 on the CSUP lines of buses 88 and 89. 
The CRC ERROR codes also are transmitted from 
crosslinks 90 and 95 to memory controllers 70' and 
75' via^ crosslinks 90' and 95' and via buses 88' 
and 89': 

Transfers of data via read and write cycles 
(transactions) will now be described. Reference - 
should be made to Figures 25A-D, which are timing 
diagrams Illustrating the transfer on 
crosslink/memory controller buses 88 and 89 of 
write Information (Figures 25A & 25B) and read 
information (Figures 25C & 25D). 

In accordance with the invention, a processor 
transmits write or read Information to a data router. 
For example, CPU module 30 transmits write in- 
formation to crosslinks 90 and 95 of 
crosslink/memory controller buses 88 and 89. 

The write or read information respectively des- 
ignates a write or read transaction. In the preferred 
embodiment, write and read transactions are iden- 
tified by the transmission of control codes from 
memory controllers 70 and 75 on the CSDN lines 



of the crosslink/merhory controller buses 88 and 
89. As shown in Figures 25A-25D, a WRITE code 
is transmitted on the CSDN lines by CPU module 
30 to uniquely designate a write transaction, and a 

5 READ code is transmitted on the CSDN lines to 
uniquely designate a read transaction. A setup 
write transaction, which has been discussed pre- 
viously, looks identical to the write transaction Illus- 
trated in Figures 25A and 25B, except a SETUP 

70 WRITE code is transmitted on the CSDN lines to 
distinguish it from a write transaction. 

In the present invention, write Information trans- 
mitted from the processor includes a write address, 
and read information transmitted from the proces- 

75 sor includes a read address. A write address des- 
ignates a destination for write data, and a read 
address designates a source for read data. As 
embodied herein and shown in Figures 25A-D, the 
most significant bits in the write or read address 

20 are included in the data transmitted on the data 
lines of the crosslink/memory controller buses 88 
and 89 at the start of the write or read transaction. 

In order to store write data in a system re- 
source (e.g., I/O module 100) or access read data 

2S from a system resource, the processor initiating the 
write or read transaction must transmit information 
to the system resource. However, the I/O modules 
in system 10 are not directly connected to CPU 
modules 30 and 30'. Instead, the I/O modules must 

30 be accessed via the system resource buses, i.e., 
module interconnects 130, 132. 130', and 132'. 

Crosslinks 90. 95, 90', and 95' function as data 
routers that set up paths coupling specific proces- 
sors to a particular system resource. However. 

35 switching logic is required In the crosslinks be- 
cause a system resource that is the source of read 
data or the destination for write data may be coup- 
led either to module interconnects 130 and 132 or 
to module interconnects 130' and 132'. 

40 After write information including a write address 
specifying a destination is transmitted, for example, 
from memory controller 70 to crosslink 90, this 
information must be forwarded to the system re- 
source corresponding to the destination. If, for ex- 

45- ample, the destination corresponds to I/O module 
100. the switching logic in crosslink 90 must be 
configured to transmit write information from mem- 
ory controller 70 to module interconnect 1 30, If. on 
the other hand, the destination corresponds to I/O 

50 module 100, the switching logic in crosslink 90 
must be configured to transmit write information 
from merinory controller 70 to module interconnect 
130' via crosslink pathway 25 and crosslink 90'. 
Similarly, if read information including a read 

55 address specifying a source is transmitted, for ex- 
ample, from memory controller 70 to crosslink 90, 
this read information must be fonwarded to the 
system resource corresponding to the source to 
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enable accessing of read data by mennory control- 
ler 70. 

Therefore, In accordance with the invention, the 
data routers include means for decoding the read 
or write address during the transaction. The data 
routers deternnine which pair of system resource 
buses, module interconnects 130 and 132 or mod- 
ule interconnects 130 and 132 , will be involved in 
the transaction. The data routers -also determine 
whether the destination (for a write transaction) or 
the source (for a read transaction) corresponds to a 
system resource coupled to the system resource 
buses. 

In the preferred embodiment of the Invention 
shown in Figures 11 and 12, decoders 971, 996, 
and 998 are provided to decode the write or read 
address transniitted during a write or read transac- 
tion to generate the internal control signals required 
to set up paths between the memory controllers 
and one pair of system resource buses. Figure 11 
shows the switching logic In crosslinks 90 and 95 
involved in the transfer of control codes between 
the CSUP and CSDN lines of crosslink/memory 
controller buses 88 and 89 and of module intercon- 
nects 130 and 132. Figure 12 shows the switching 
logic in crosslink 90 involved In the transfer of data, 
such as write and read addresses and write and 
read data, between the bidirectional data lines of 
crosslink/memory controller buses 88 and 89 and 
of module interconnects 130 and 132. 

As embodied herein, one bit in the read or 
write address transmitted during the read or write 
transaction identifies the zone in which the source 
or destination is located. As a result, the decoders 
in the crosslink can determine which pair of module 
interconnects will be involved in the transaction. 
Furthermore, if the read or write address con-e- 
sponds to an address in the I/O space, of system 
10, the decoders in the crosslink can determine 
that the transaction will Involve a system resource, 
i.e., an I/O module, coupled to. one pair of the 
module Interconnects. 

In response to initiation of a read or write 
transaction, the switching logic in crosslinks i90, 
90', 95, and 95' Is configured in accordance with 
the decoded read or write address. 

When system 10 Is operating in duplex mode, 
the data switching logic in the crosslinks is config- 
ured to forward data, i.e., read addresses, write 
addresses, or write data, to system resource buses 
130 and 132. if the transaction involves the pair of 
system resource buses 130 and 132 in zone 11 
and if the source or destination corresponds to a 
system resource coupled to the system resource 
buses. In this example, a path is configured to 
forward data to module interconnect 130 either 
from CPU 40 via crosslink 90 or from CPU AO via 
crosslinks 90' and 90. and also to forward data to 
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module interconnect 132 either from CPU 50 via 
cross link 95 or from CPU 50' via crosslinks 95 
and 95. 

The exact configuration of the switching logic 

5. shown in Figures 11 and 12 In order to set up this 
path to module interconnects 130 and 132. is de- 
scribed above in the description relating to a third 
example of DMA transfers, in which DMA data is 
forwarded to system resource buses 130 and 132. 

70 If the transaction involves the pair of system 
resource buses 130' and 132' in zone 1l' and if 
the source or destination corresponds to a system 
resource coupled to the system resource buses, 
the data switching logic in the crosslinks is config- 

75 ured to forward the data to system resource buses 
130' and 132'. In this second example, a path is 
configured to forward data to module interconnect 
130' either from CPU 40' via crosslink 90' or from 
CPU 40 via crosslinks 90 and 90'. and also to 

20 forward data to module interconnect 132 either 
from CPU 50' via crosslink 95' or from CPU 50 via 
crosslinks 95 and 95'. 

The exact configuration of the switching logic 
shown in Rgures 11 and 12 in order to set up this 

25 path to module interconnects 130' and 132' Is 
described above In the description reiatlng to a 
fourth example of DMA transfers, in which DMA 
data is forwarded to system resource buses 130' 
and 132'. 

30 In the preferred embodiment of the invention, 
the read or write address transmitted to the cros- 
slinks at the beginning of the transaction is needed 
In order to configure the switching logic. As a 
result, there Is not sufficient time to decode the 

35 address, configure the switching logic, and then 
fonward that same address to the source or des- 
tination via the switching logic. 

However, the read or write address provided 
by a processor at the beginning of a transaction 

40 must be transmitted to the system resource cor- 
responding to the source or destination. The read 
address Is required because it allows the system 
resource to access read data from the source 
specified by the read address, and the write ad- 

45 dress Is necessary to enable the system resource 
to store write data in the destination designated by 
the write address. 

One way of implementing the system, which 
would result in the read or write address being 

50 available for forwarding after the switching logic is 
configured, would be to latch all data entering the 
cross links for a time period long enough to config- 
ure the switching logic. However, this would require 
the addition of circuitry to the system for holding all 

55 data transmitted from the processors to the system 
resources. Furthermore, the pipeline delay between 
components in the system would be increased, 
unless other logic also was added to recognize and 
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delay only addresses sent at the beginning of a 
read or write transactions. 

In computer system 10. the read and write 
addresses are forwarded from the processors to 
the system resources without the addition of any 
special circuitry to the crosslinks and without in- 
creasing the delay required before data transmitted 
from a processor is received at an 1/0 module. 

In accordance with the invention, a read or 
write address is retransmitted from the processors 
to the data routers during a read or write transac- 
tion. As shown in Rgures 25A-D, the most signifi- 
cant bits of the read or write address, which are 
transmitted on the CSDN lines of buses 88 and 89 
at the beginning of the transaction, are then retran- 
smitted on the CSDN lines. 

In the present invention, the read or write ad- 
dress that is retransmitted by the processors is 
then forwarded to the appropriate pair of system 
resource buses involved in the transaction, pro- 
vided the source or destination corresponds to a 
system resource (e.g., an I/O module) coupled to 
one of the pairs of system resource buses. The 
address is forwarded to module interconnects 130 
.and 132, or to module interconnects 130' and 132', 
via the switching logic in crosslinks 90. 95^ 90', 95'. 

The retransmission of the address provides the 
crosslinks with sufficient time to set up a path from 
specific processors to the system resource cor- 
responding to the source or destination. As a re- 
sult, when the. address is retransmitted, the switch- 
ing logic is configured to route the address to the 
pair of module interconnects coupled to the system . 
resource. 

In accordance with the invention, after trans- 
mission, of a write address, the processors transmit 
write data to the data routers coupled to each 
processor. The switching logic in the crosslinks is 
properly configured, and therefore this write data is 
forwarded via the switching logic to the appropriate 
pair of system resource buses involved in the 
transaction. The manner in which write data is 
forwarded by the switching logic is the same as the 
manner in which the retransmitted write address is 
forwarded. 

As shown in Rgures 25A and 25B, write data is 
provided during a write transaction by memory 
controllers 70 and 75 on the bidirectional data lines 
of buses 88 and 89. The system resource that 
receives the write address and write data responds 
by attempting to store the write data in the destina- 
tion designated by the write address. 

In accordance with the invention, after forward- 
ing of the read address by the switching logic 
during a read transaction, the data switching logic 
in the data routers is then reconfigured in accor- 
dance with the read address decoded by the data 
routers. As shown in Rgure 12, the data switching 



logic is reconfigured after the read address is sent 
in the down direction in order to allow read data to 
be forwarded to CPU modules 30 and 30' in the up 
direction. 

5 When system 10 Is operating in duplex mode, 
the data switching logic in the crosslinks is config- 
ured to forward read data from system resource 
buses 130 and 132, If the transaction involves the 
pair of system resource buses 130 and 132 in zone 

70 11 and if the source corresponds to a system 
resource coupled to the system resource buses. In 
this example, a path is configured to forward read 
data from module interconnect 130 to CPU 40 via 
crosslink 90 and to CPU 40' via crosslinks 90 and 

is 90\ and also to forward read data from module 
interconnect 132 to CPU 50 via crosslink 95 and to 
CPU 50' via crosslinks 95 and 95'. 

The exact configuration of the data switching 
logic shown In Rgure 12 in order to set up this 

20 path from module interconnects 130 and 132 is 
described above in the description relating to a first 
example of DMA transfers, in which DMA data is 
fon/varded from system resource • buses 130 and 
132. 

25 If the transaction involves the pair of system 
resource buses 130' and 132' in zone 1l' and if 
the source corresponds to a system resource 
coupled to the system resource buses, the data 
switching logic In the crosslinks is configured to 

30 forward the read data from system resource buses 
1 30 and 1 32 . In this second example, a path is 
configured to forward read data from module inter- 
connect 130' to CPU 40' via crosslink 90' and to 
CPU 40 via crosslinks 90' and 90, and also to 

35 forward read data from module interconnect 132' to 
CPU 50' via crosslink 95' and to CPU 50 via 
crosslinks 95' and 95. 

The exact configuration of the data switching 
logic shown in Rgure 12 in order to set up this 

40 path from module interconnects 130' and 132* is 
described above in the description relating to a 
second example of DMA transfers, in which DMA 
data is forwarded from system resource buses 130' 
and 132. 

45 In response to receipt of the read address, a 
system resource accesses read data from the 
source designated by the read address, and trans- 
mits this read data to the pair of system resource 
buses coupled to the system resource. The data 

50 switching logic in the crosslinks is reconfigured 
after forwarding of the read address. Therefore, in 
duplex mode the read data is received by each of 
the crosslinks and is fonvarded via the switching 
logic to each of the processors. 

55 As shown in Rgure 25C, in response to initi- 
ation of a read transaction, read data is transmitted 
to memory controllers 70, 75. 70'. and 75' by 
crosslinks 90, 95. 90', and 95' on the bidirectional 
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data lines of buses 88. 89. 88', and 89'. As a 
result. CPU's 40, 50. 40', and 50' In CPU modules 
30 and 30' are able to access read data located at 
the source designated by the read address. 

Preferably, as shown in Rgures 25A and 25C, 
in response to transmission of a write address and 
write data or a read address to one of the system 
resources (e.g., I/O module 100), an acknowledge 
code Is transmitted to each of the processors on 
the CSUP lines of module interconnects 130, 132, 
130'. and 132' and on the CSUP lines of buses 88. 
89, 88'. and 89'. The system resource that re- 
ceives read or write infonmation from the proces- 
sors will transmit an ACK control code to each of 
the processors to indicate successful transmission 
of the write address and write data or of the read 
address. 

As an example, if I/O module 100 receives the 
write address and write data or the read address, 
firewalls 1000 and 1010 in. I/O module 100 transmit 
an ACK control code to the crosslinks 90 and 95 
via the pair of module interconnects 130 and 132 to 
indicate that firewalls 1000 and 1010 have not 
detected an error in data transmission from the 
processors in the down direction. In CPU module 
30, memory controllers 70 and 75 receive the ACK 
codes from crosslinks 90 and 95 on the unidirec- 
tional control lines CSUP of buses 88 and 89. The 
ACK codes also are transmitted from crosslinks 90 
and 95 to memory controllers 70' and 75* in CPU 
module 30 via crosslinks 90' and 95' and via 
buses 88' and 89'. 

Preferably, as shown in Figures 25B and 25D. 
in response to transmission of a write address and 
write data or a read address to one of the system ^ 
resources (e.g., I/O module 100), an error code is 
transmitted to each of the processors on the CSUP 
lines of module Interconnects 130, 132. 130', and 
132' and on the CSUP' lines of buses 88.. 89, 88'. ' 
and 89'. The system resource that receives read or 
write Information from the processors will transmit 
an ERROR control code to each of the processors 
to indicate unsuccessful transmission of the write 
address and write data or of the read address. 

As another example, if I/O module 100 receives 
the write address and write data or the read ad- 
dress, firewalls 1000 and 1010 in I/O module 100 
transmit an ERROR control code to the crosslinks 
90 and 95 via the pair of module interconnects 130 
and 132 to indicate that firewalls 1000 and 1010 
have detected an error in data transmission from 
the processors in the down direction. In the pre- 
ferred embodiment of system 10, the ERROR con- 
trol code Indicates that firewalls 1000 and 1010 
have detected a miscompare error as a result of 
comparing the data transmitted from one of the 
CPU modules on module interconnect 130 with the 
data transmitted from the other CPU module on 



' module interconnect 132. In CPU module 30. mem- 
ory controllers 70 and 75 receive the ERROR con- 
trol codes from crosslinks 90 and 95 on the un- 
idirectional control lines CSUP of buses 88 and 89. 

5 The ERROR control codes also are transmitted 
from crosslinks 90 and 95 to memory controllers 
70' and 75' in CPU module 30' via crosslinks 90' 
and 95' and via buses 88' and 89'. 

Preferably, during read and write transactions, 

70 ■ including a write transaction used to transmit DMA 
starting address Information, and during a setup 
write transaction used to transmit the DMA count, 
read or write address information is interleaved on 
the bidirectional data lines of crosslink/memory 

75 controller buses 88 and 89 with byte mask informa- 
tion. As shown In Rgures 25A and 25C, four bits of 
byte mask data are provided for each longword 
(i.e., four bytes) of data. As a result, particular 
bytes within an addressed longword can be se- 

20 lected using the byte mask data. 

In the preferred embodiment of the invention, 
certain transfers of information via DMA occur in 
.synchronism with a particular clock phase. -As 
shown in Rgures 24A-E, whenever DMA data is 

25 transferred in either direction, the beginning of the 
transmission of each longword is synchronized with 
clock phase P6, which is generated by phase gen- 
erator 2310 (Rgure 23). Furthermore, DONE and 
. ..ERROR codes are always transmitted to the mem- 

30 cry controllers on the CSUP lines in phase with 
clock phase P6. 

In the embodiment of the Invention shown In 
Rgures 24A-E, an ACK code is transmitted by the 
firewalls on the CSUP lines to the memory contrbl- 

35 lers in synchronism with clock phase P6. When this 
occurs, transmission of the ACK code is followed 
by transmission of a STALL code, because the 
bidirectional data lines of crosslink/ memory con- 
troller buses 88 and 89 only cany sixteen bits of 

40 data, whereas the memory controllers can process 
data in units of thirty-two bits. Alternatively, if the 
ACK code is not transmitted in phase with the 
clock phase P6. the subsequent STALL code Is not 
necessary, but the CSUP lines must be idle during 

45 the preceding clock phase P6. 

It will be apparent to those skilled in the art that 
various modifications and variations can be made 
in the data transfer protocol of the present inven- 
tion without departing from the scope or spirit of 

50 the invention. Thus, It is intended that the present 
invention cover the modifications and variations or 
this invention provided they come within the scope 
of the appended claims and their equivalents. 

55 

Claims 

1. A process for transferring data via DMA between 
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components in a computer system, wherein the 
computer system includes a memory controller, a 
, first system resource bus coupled to a system 
resource, a second system resource bus coupled 
to a system resource, and a data router, wherein 
the data router includes switching logic for coupling 
the memory controller to the first system resource 
bus or to the second system resource bus, and 
wherein the process comprises the following steps: 
transmitting DMA setup information designating a 
setup write transaction, from the memory controller, 
to one of the system resources, via the data router 
and via the system resource bus coupled to said 
system resource, wherein said DMA setup Informa- 
tion corresponds to a DMA transfer, and indicates a 
direction of the DMA transfer as either an up direc- 
tion from said system resource to said memory 
controller or a down direction from said memory 
controller to said system resource, indicates which 
system resource bus the DMA transfer will Involve, 
and indicates whether said DMA transfer will In- 
volve a system resource coupled to one of the 
system resource buses; 

decoding said DMA setup Iriformation in the data 
router during said setup write transaction, to deter- 
mine the direction of said DMA transfer, to deter- 
mine which system resource bus said DMA trans- 
fer will involve, and to determine whether said DMA 
transfer will involve a system resource coupled to 
one of the system resource buses; 
transmitting a start DMA code, from said memory 
controller, to said system resource, via the data 
router and via the system resource bus coupled to 
said system resource; 

configuring tiie switching logic In said data router, 
in response to transmission .of said DMA start 
code, and in accordance with said DMA setup 
information decoded by said data router; 
wherein said switching logic is configured to for- 
ward DMA data, from the first system resource bus 
to said memory controller, at times when said DMA 
transfer is in the up direction and said DMA trans- 
fer will , involve the first system resource bus and 
said DMA transfer will involve a system resource 
coupled to. one of the system resource buses; 
wherein said switching logic is configured to for- 
ward DMA data, from the second system resource 
bus to said memory controller, at times when said 
DMA transfer is in the up direction and said DMA 
transfer will involve the second system resource 
bus and said DMA transfer will involve a system 
resource coupled to one of the system resource 
buses; 

wherein said switching logic is configured to for- 
ward DMA data, from said memory controller, to 
the first system resource bus, at times when said 
DMA transfer is in the down direction and said 
DMA transfer will involve the first system resource 



bus and said DMA transfer will involve a system 
resource coupled to one of. the system resource 
buses; and 

wherein said switching logic is configured to for- 
5 ward DMA data, from said memory controller, to 
the second system resource bus, at times^ when 
said DMA transfer is in the down direction and said 
DMA transfer will involve the second system re- 
source bus and said DMA transfer will involve a 
10 system resource coupled to one of the system 
resource buses; 

transmitting an aci<nowledge code, from said sys- 
tem resource, to said memory controller, via the 
system resource bus coupled to said system re- 
is source, and via the switching logic in said data 
router, in response to transmission of said DMA 
start code; 

transmitting DMA data, after transmission of said 
acknowledge code, at times when said DMA trans- 

20 fer is in tiie up direction, from said system re- 
source, to said memory controller, via the- system 
resource bus coupled to said system resource, and 
via the switching logic in said data router; 
transmitting DMA data, after transmission of said 

26 acknowledge code, at times when said DMA trans- 
fer Is in the down direction, from said memory 
controller, to said system resource, via the switch- 
ing logic in said data router, and via tiie system 
resource bus coupled to said system resource; and 

30 transmitting a done . code, after transmission of 
DMA data, from said system resource, to said 
memory controller, via the system resource bus 
coupled to said system resource, and via the 
switching logic in said data router, to Indicate suc- 

35 cessful DMA data transmission. 

2. A process in accordance witii claim 1, and 
furtiier comprising: 

transmitting an error code, frorn the switching logic 
in said data router, to said memory controller, after 
40 transmission of said acknowledge code, to indicate 
unsuccessful DMA data transmission. 

3. A process in accordance with claim 2. and 
further comprising: 

transmitting a stall code, from said system re- 
45 source, to said memory controller, via the system 
resource bus coupled to said system resource, and 
via the switching logic in said data router, after 
transmission of said acknowledge code, to indicate 
said system resource is not ready to accept DMA 
50 data transmitted in the down direction from said 
memory controller and to indicate said system re- 
source is not ready to send DMA data in the up 
direction. 

4. A process in accordance witii claim 1, and 
55 further comprising: 

transmitting an error code, from said system re- 
source, to said memory controller, via the system 
resource bus coupled to said system resource, and 
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via the switching logic in said data router, after 

transmission of DMA data in the up direction, to 

indicate an error in DMA data transmission. 

5. A process in accordance with claim 4, in which 

the error code is a CRC enror code to indicate a 

CRC error in DMA data transmission. 

6: A process in accordance with claim 5, and 

further comprising: 

transmitting a stall code, from said system re? 
source, to said memory controller, via the system 
resource bus coupled to said system resource, and 
via the switching logic in said data router, after 
transmission of said acknowledge code, to indicate 
said system resource is not ready to send DMA 
data in the up direction. 

7. A process in accordance with claim 4, and 
further comprising: 

transmitting a . stall code, from said system re- 
source, to said memory controller, via the system 
resource bus coupled to said system resource, and 
via the switching logic in said data router, after 
transmission of said acknowledge code, to Indicate 
said system resource is not ready to send DMA 
data in the up direction. 

8. A process in accordance with claim .1, and 
further comprising: 

transmitting a stall code, from said system re- 
source, to said memory controller, via the system 
resource bus coupled to said system resource, and 
via the switching logic in said data router, after 
transmission of said acknowledge code, to indicate 
said system resource is not ready to accept DMA 
data transmitted in the down direction from said 
memory controller and to indicate said system re- 
source is not ready to send DMA data in the up 
direction. 

9. A process in accordance with claims 1. 2. 3, 4, 
7, or 8, wherein said system resource has a pointer 
register for indicating a DMA data address to be 
accessed by said system resource, and further 
comprising: 

transmitting DMA starting address information dur- 
ing a write transaction, from the memory controller, 
to said system resource, via the data router and via 
the system resource bus coupled to said system 
resource, prior to transmission of said start DMA 
code from said memory controller to said system 
resource, . wherein said DMA starting address in- 
formation specifies a DMA data address to be 
accessed by said system resource during said 
DMA transfer; 

storing the DMA data address, in the pointer regis- 
ter in said system resource, after transmission of 
said DMA starting address information from said 
memory controller to said system resource. 

10. A process in accordance with claim 9, wherein 
said system resource has a count register for in- 
dicating the number of bytes of DMA data to be 



accessed by said- system resource during said 
DMA transfer, and further comprising: 
transmitting DMA count information during a write 
transaction, from the memory controller, to said 

5 system resource, via the data router and via the 
system resource bus coupled to said system re- 
source, prior to transmission of said start DMA 
code from said memory controller to said system 
resource, wherein said DMA count information 

10 specifies a DMA count of the number of bytes of 
DMA data to be accessed by said system resource 
during said DMA transfer; 

storing the DMA count, in the count register In said 

system resource, after transmission of said DMA 
IS count information from said memory controller to 
said system resource. 

11. A process in accordance with claims 1, 2, 3, 4, 
7, or 8, wherein said system resource has a count 
register for indicating the number of bytes of DMA 

20 data to be . accessed by said system resource 
during said DMA transfer, and further comprising: 
transmitting DMA count information during a write 
transaction, from the memory controller, to said 
system resource, via the data router and via the 

25 system resource bus coupled to said system re- 
source, prior to transmission of said start DMA 
code from said memory controller to said system 
resource, wherein said DMA count information 
specifies a DMA count of the number of bytes of 

30 DMA data to be accessed by said system resource 
during said DMA transfer; 

storing the DMA count, in the count register in said 
system resource, after transmission of said DMA 
count Infomnatipn from said memory controller to 
35 said system resource. 

1 2. A process in accordance with claim 1 1 . wherein 
said switching logic includes downstream and up- 
stream switching logic, and wherein the process 
comprises: 

40 configuring the downstream and upstream switch- 
ing logic in said data router, in response to trans- 
mission of said DMA start code, and in accordance 
with said DMA setup information decoded by said 
data router; 

45 wherein said upstream switching logic is configured 
to forward codes, from the first system resource 
bus to said memory controller, at times when said 
DMA transfer will involve the first system resource 
bus and said DMA transfer will involve. a system 

50 resource' coupled to one of the system resource 
buses; 

wherein said upstream switching logic is configured 
to forward codes, from the second system resource 
bus to said memory controller, at times when said 
55 DMA transfer will involve the second system re- 
source bus and said DMA transfer will involve a 
system resource coupled to one of the system 
resource buses; 
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wherein said upstream switching logic is configured 
to forward DMA data, from the first system re- 
source bus to said memory controller, at times 
when said DMA transfer is in the up direction and 
said DMA transfer will involve the first system 
resource bus and said DMA transfer will Involve a 
system resource coupled to one of the system 
resource buses; 

wherein said upstream switching logic Is configured 
to forward DMA data, from the second system 
resource bus to said memory controller, at times 
when said DMA transfer is in the up direction and 
said DMA transfer will involve the second system 
resource bus and said DMA transfer will involve a 
system resource coupled to one of the system 
resource buses; 

wherein said downstream switching logic is config- 
ured to forward DMA data, from said memory con- 
troller, to the first system resource bus, at times 
when said DMA transfer is in' the. down direction 
and said DMA transfer will involve the first system 
resource bus and said DMA transfer will involve a 
system resource coupled to one of the system 
resource buses; and 

wherein said downstream switching logic is config- 
ured to forward DMA data, from said memory con- 
troller, to the second system resource bus, at times 
when said DMA transfer is in the down direction 
and said DMA transfer will involve the second 
system resource bus and said DMA transfer will 
involve a system resource coupled, to one of the 
system resource buses. 

13. A process in accordance with claim 9, wherein 
said switching logic includes downstream and up- 
stream switching logic, and wherein the process 
comprises: 

configuring the downstream and upstream switch- 
ing logic in said data router, in response to trans- 
mission of said DMA start code, and in accordance 
with said DMA setup information decoded by said 
data router; 

wherein said upstream switching logic is configured 
to forward codes, from the first system resource 
bus to said memory controller, at times when said 
DMA transfer will involve the first system resource 
bus and said DMA transfer will involve a system 
resource coupled to one of the system resource 
buses; 

wherein said upstream switching logic is configured 
to forward codes, from the second system resource 
bus to said memory controller, at times when said 
DMA transfer will involve the second system re- 
source bus and said DMA transfer will involve a 
system resource coupled to one of the system 
resource buses; 

wherein said upstream switching logic is configured 
to forward DMA data, from the first system re- 
source bus to said memory controller, at times 



when said DMA transfer is in the up direction and 
said DMA transfer will involve the first system 
resource bus and said DMA transfer will involve a 
system resource coupled to one of the system 

5 resource buses; 

wherein said upstream switching logic is configured 
to fonvard DMA data, from the second system 
resource bus to said memory controller, at times 
when said DMA transfer is in the up direction and 

10 said DMA transfer will involve the second system 
resource bus and said DMA transfer will involve a 
system resource coupled to one of the system 
resource buses; . 

wherein said downstream switching logic is config- 
75 ured to forward DMA data, from said memory con- 
troller, to the first system resource bus, at times 
when said DMA transfer is in the down direction 
and said DMA transfer will involve the first system 
resource bus and said DMA transfer will involve a 
20 system resource coupled to one of the system 
resource buses; and 

wherein said downstream switching logic is config- 
ured to forward DMA data, from said memory con- 
troller, to the second system resource bus, at times 

25 when said DMA transfer is in the down direction 
and said DMA transfer will involve the second 
system resource bus and said DMA transfer will 
involve a system resource coupled to one of the 
system resource buses. 

30 14. A process in accordance with claims 1, 2, 3, 4, 
7. or 8. wherein said switching logic includes down- 
stream and upstream switching logic, and wherein 
the process comprises: 

configuring the downstream and upstream switch- 
35 ing logic in said data router, in response to trans- 
mission of said DMA start code, and In accordance 
with said DMA setup information decoded by said 
data router; 

wherein said upstream switching logic is configured 
40 to forward codes, from the first system resource 
bus to said memory controller, at times when said 
DMA transfer will involve the first system resource 
bus and said DMA transfer will involve a system 
resource coupled to one of the system resource 
45 buses; 

wherein said upstream switching logic is configured 
to forward codes, from the second system resource 
bus to said memory controller, at times when said 
DMA transfer will involve the second system re- 
50 source bus and said DMA transfer will involve a 
system resource coupled to one of the system 
resource buses; 

wherein said upstream switching logic Is configured 
to forward DMA data, from the first system re- 
55 source bus to said memory controller, at times 
when said DMA transfer is in the up direction and 
said DMA transfer will involve the first system 
resource bus and said DMA transfer will involve a 
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system resource coupled to one of the system 
resource buses; 

wherein said upstream switching logic is configured 
to forward DMA data, from the second system 
resource bus to said memory, controller, at times 
when said DMA transfer is In the up direction and 
said DMA transfer will Involve the second system 
resource bus and said DMA transfer will involve a 
system resource coupled to one of the system 
resource buses; 

wherein said downstream switching logic is config- 
ured to fonward DMA data, from said memory con- 
troller, to the first system resource, bus, at times 
when said DMA transfer is in the down direction 
and said DMA transfer will involve the first system 
resource bus and said DMA transfer will involve a 
system resource coupled to one of the system 
resource buses; and 

wherein said downstream switching logic is config- 
ured to forward DMA data, from said memory con- 
troller, to the second system resource bus, at times 
when said DMA transfer is in the down direction 
and said DMA transfer will involve the second 
system resource bus and said DMA transfer will 
involve a system resource coupled to one of the 
system resource buses. 

1 5. A process for transferring data via DMA be- 
tween components In a conriputer system, wherein 
the computer system includes a first memory con- 
troller, a second memory controller, a first system 
resource bus coupled to a system resource, a 
second system resource bus coupled to another 
system resource, a. fir^t data router coupled to the 
first memory controller, to the first system resource 
bus, and to the second system resource bus. and a 
second data router coupled to the first memory 
controller, to the first system resource bus, and to 
the second system resource bus, wherein each of 
the data routers includes switching logic, and 
wherein the process comprises the following steps: 
transmitting DMA setup information designating a 
setup write transaction, from a memory controller, 
to one of the system resources, via the data router 
coupled to the memory controller and via the sys- 
tem resource bus coupled to said system resource, 
wherein said DMA setup information corresponds 
to a DMA transfer, and indicates a direction of the 
DMA transfer as either an up direction from said 
system resource to said memory controller or a 
down direction from said memory controller to said 
system resource, indicates which system resource 
bus the DMA transfer will involve, and indicates 
whether said DMA transfer will involve a system 
resource coupled to one of the system resource 
buses; 

decoding said DMA setup information in the data 
router coupled to said memory controller during 
said setup write transaction, to determine the direc- 



tion of said DMA transfer, to determine which sys- 
tem resource bus said DMA transfer will Involve, 
and to determine whether said DMA transfer will 
involve a system resource coupled to one of the 

5 system resource buses; 

transmitting a start DMA code, from said memory 
controller, to said system resource, via the data 
router coupled to said memory controller and via 
the system resource bus coupled to said system 

10 resource; 

configuring the switching logic in said data router, 
in response to transmission of said DMA start 
code, and in accordance with said DMA setup 
information decoded by said data router; 

75 wherein said switching logic is configured to for- 
ward DMA data, from the first system resource bus 
to said memory controller, at times when said DMA 
transfer is in the up direction and said DMA trans- 
fer will involve the first system resource bus and 

20 said DMA transfer will Involve a system resource 
coupled to one of the system resource buses; 
wherein said switching logic is configured to for- 
ward DMA data, from the second system resource 
bus to said memory controller, at times when said 

25 DMA transfer is in the up direction and said DMA 
transfer will involve the second system resource 
bus and said DMA transfer will involve a system 
resource coupled to one of the system resource 
buses; 

30 wherein said switching logic is configured to for- 
ward DMA data, from said memory controller, to 
the first system resource bus. at times when said 
DMA transfer Is In the down direction and said 
DMA transfer will involve the first system resource 

35 bus and said DMA transfer will involve a system 
resource coupled to one of the system resource 
buses; and 

wherein said switching logic is configured to for- 
ward DMA .data, from said memory controller, to 

40 the second system resource bus, at times when 
said DMA transfer is in the down direction and said 
DMA transfer will involve the second system re- 
source bus and said DMA transfer will involve a 
system resource coupled to one of the system 

45 resource buses; 

transmitting an acknowledge code, from said sys- 
tem resource, to said memory controller, via the 
system resource bus coupled to said system re- 
source, and via the switching logic in said data 

50 router. In response to transmission of said DMA 
start code; 

transmitting DMA data, after transmission of said 
acknowledge code, at times when said DMA trans- 
fer is in the up direction, from said system re- 
55 source, to said memory controller, via the system 
resource bus coupled to said system resource, and 
via the switching logic in said data router; 
transmitting DMA data, after transmission of said 
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acknowledge code, at times when said DMA trans- 
fer is in the down direction, from said memory 
controller, to said system resource, via the switch- 
ing logic in said data router, and via the system 
resource bus coupled to said system resource; 5 
transmitting a done code, after transmission of 
DMA data, from said system resource, to said 
memory controller, via the system resource bus 
coupled to said system resource, and via the 
switching logic in said data router, to indicate sue- 10 
cessful DMA data transmission. 
16. A process in accordance with claim 15, wherein 
the first data router is coupled to the second sys- 
tem resource bus via the switching logic in the 
second data router, and the second data router is 75 
coupled to the first system resource bus via the 
switching logic in the first data router, and wherein 
the process comprises the following steps: 
transmitting the same DMA setup information, from 
each of the memory controllers, to one of the- 20 
system resources, via the first and second data 
routers and via the system resource bus coupled to 
said system resource; 

decoding said DMA setup information In the first 
and second data routers; 25 
transmitting a start DMA code, from each of said 
memory controllers, to said system resource, via 
the first and second data routers and via the sys- 
tem resource bus coupled to said system resource; 
configuring the switching logic In the first and sec- 30 
ond data routers, and in accordance with said DMA 
setup information decoded by the first and second 
data routers; 

wherein said switching logic Is configured to for- 
ward DMA data, from the first system resource 35 
bus, via the switching logic In the first data router 
to the first memory controller, and via the switching 
logic in the first and second data routers to the 
second memory controller, at times when said 
DMA transfer is in the up direction and said DMA 40 
transfer will involve the. first system resource bus 
and said DMA transfer will Involve a system re- 
source coupled to one of the system resource 
buses; 

wherein said switching logic is configured to for- 4S 
ward DMA data, from the second system resource 
bus, via the svyltching logic in the second data 
router to the second memory controller, and via the 
switching logic in the second and first data routers 
to the first memory controller, at times when said so 
DMA transfer is in the up direction and said DMA 
transfer will involve the second system resource 
bus and said DMA transfer will involve a system 
resource coupled to one of the system resource 
buses; 55 
wherein said switching logic is configured to for- 
ward DMA data, from the first memory controller 
via the switching logic in the first data router, and 



fi'om the second memory controller via the switch- 
ing logic in the second and first data routers, to the 
first system resource bus. at times when said DMA 
transfer Is in the down direction and said DMA 
transfer will involve the first system resource bus 
and said DMA transfer will involve a system re- 
source coupled to one of the system resource 
buses; and 

wherein said switching logic is configured to for- 
ward DMA data, from the first memory controller 
via the switching logic in the first and second data 
routers, and from the second memory controller via 
the switching logic in the second data router, to the 
second system resource bus, at times when said 
DMA transfer is in the down direction and said 
DMA transfer will involve the second system re- 
source bus and said DMA transfer will involve a 
system resource coupled to one of the system 
resource buses; 

transmitting an acknowledge code, from said sys- 
tem resource, to each of said memory controllers, 
via the system resource bus coupled to said sys- 
tem resource, and via the switching logic in the first 
and second data routers; 

transmitting DMA data, at times when said DMA 
transfer is in the up direction, from said system 
resource, to each of said memory controllers, via 
the system resource bus coupled to said system 
resource, and via the switching logic in the first and 
second data routers: 

transmitting the same DMA data, at times when 
said DMA transfer is in the down direction, from 
each of said memory controllers, to said system 
resource, via the switching logic In the first and 
second data routers, and via the system resource 
bus coupled to said system resource; 
transmitting a done* code, from said system re- 
source, to each of said memory controllers, via the 
system resource bus coupled to said system re- 
source, and via the switching logic in the first and 
second data routers. 

17. A process In accordance with claim 16 and 
further comprising: 

transmitting an error code, from the switching logic 
In the first and second data routers, to each of said 
memory controllers, after transmission of said ac- 
knowledge code, to indicate unsuccessful DMA 
data transmission. 

18. A process in accordance with claim 17, and 
further comprising: 

transmitting a stall code, from said system re- 
source, to each of said memory controllers, via the 
system resource bus coupled to said system re- 
source, and via the switching logic in the first and 
second data routers, after transmission of said ac- 
knowledge code, to indicate said system resource 
is not ready to accept DMA data transmitted in the 
down direction from each of said memory controi- 
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iers and to indicate said system resource is not 
ready to send DMA data in the up direction. 

19. A process in accordance with clainr) 16, and 
further comprising: 

transmitting an error code, from said system re- 
source, to each of said memory controllers, via the 
system resource bus coupled to said system re- 
source, and via the switching logic in the first and 
second data routers, after transmission of DIVIA 
data In the up direction, to indicate an error in DMA 
data transmission. 

20. A process in accordance with claim 19, in 
which the eror code is a CRC error code to 
indicate a CRC error in DMA data transmission. 

21. A process in accordance with claim 20. and 
further comprising: 

transmitting a stall code, from said system re- 
source, to each of said memory controllers, via the 
system resource bus coupled to said system re- 
source, and via the switching logic in the first and 
second data routers, after transmission of said ac- 
knowledge code, to indicate said system resource 
is not ready to send DMA data in the up direction. 

22. A process in accordance with claim 19. and 
further comprising: 

transmitting a stall code, from said system re- 
source, to each of said memory controllers, via the 
system resource bus coupled to said system re- 
source, and via the switching logic in the first and 
second data routers, after transmission of said ac- 
knowledge code, to Indicate said system resource 
is not ready to send DMA data in the up direction. 

23. A process in accordance with claim 16, and 
further comprising; 

transmitting a stall code, from said system re- 
source, to each of said memory controllers, via the 
system resource bus coupled to said system re- 
source, and via the switching logic in the first and 
second data routers, after transmission of said ac- 
knowledge code, to Indicate said system resource 
is not ready to accept DMA data transmitted in the 
down direction from each of said memory control- 
lers and to indicate said system resource is not 
ready to send DMA data in the up direction. 

24. A process in accordance with claims 16, 17. 18, 
19, 22, or 23, wherein said system resource has a 
pointer register for Indicating a DMA data address 
to be accessed by said system resource, and 
further comprising: 

transmitting DMA starting address information dur- 
ing a write transaction, from each of said memory 
controllers, to said system resource, via the first 
and second data routers and via the system re- 
source bus coupled to said system resource, prior 
to transmission of said start DMA code from each 
of said memory controller to said system resource, 
wherein said DMA starting address information 
specifies a DMA data address to be accessed by 



said system resource during said DMA transfer; 
storing the DMA data address, in the pointer regis- 
ter In said system resource, after transmission of 
said DMA starting address Information from each of 
5 said memory controllers to said system resource. 

25. A process in accordance with claim 24, wherein 
said system resource has a count register for in- 
dicating the number of bytes of DMA data to be 
accessed by said system resource during said 

70 DMA transfer, and further comprising: 

transmitting DMA count information during a write 
transaction, from each of said memory controllers, 
to said system resource, via the first and second 
data routers and via the system resource bus 

75 coupled to said system resource, prior to transmis- 
sion of said start DMA code from each of said 
memory controllers to said system resource, 
wherein said DMA count information specifies a 
DMA count of the number of bytes of DMA data to 

20 be accessed by said system resource during said 
DMA transfer; 

storing the DMA count, in the count register in said 
system resource, after transmission of said DMA 
count information from each of said memory con- 
25 trollers to said system resource. 

26. A process In accordance with claims 16, 17, 18, 
19, 22, or 23. wherein said system resource has a 
count register for indicating the number of bytes of 
DMA data to be accessed by said system resource 

30 during said DMA transfer, and further comprising: 
transmitting DMA count information during a write 
transaction, from each of said memory controllers, 
to said system resource, via the first and second 
data routers and via the system resource bus 

35 coupled to said system resource, prior to transmis- 
sion of said start DMA code from each of said 
memory controllers to said system resource, 
wherein said DMA count information specifies a 
DMA count of the number of bytes of DMA data to 

40 be accessed jsy said system resource during said 
DMA transfer; . 

storing the DMA count. In the count register in said 
system resource, after transmission of said DMA 
count information from each of said memory con- 
4S trollers to said system resource. 

27. A process In accordance with claim 26, wherein 
said switching logic includes downstream and up- 
stream switching logic, and wherein the process 
comprises: 

60 configuring the downstream and upstream switch- 
ing logic in the first and second data routers, in 
response to transmission of said DMA start code, 
and in accordance with said DMA setup information 
. decoded by the first and second data routers; 

55 wherein said upstream switching logic is configured 
to forward codes, from the first system resource 
bus, via the first data router to said first memory 
controller, and via the first and second data routers 
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to said second memory controller, at times when 
said DMA transfer will involve the first system 
resource bus and said DMA transfer will involve a 
system resource coupled to one of the system 
resource buses; s 
wherein said upstream switching logic Is configured 
to forward codes, from the second system resource 
bus, via the second data router to said second 
memory controller, and via the second and first - 
data routers to said, first memory controller, at io 
times when said DMA transfer will Involve the sec- 
ond system resource bus and said DMA transfer 
will involve a system resource coupled to one of 
the system resource buses; 

wherein said upstream switching logic is configured 76 
to forward DMA data, from the first system re- 
source bus, via the first data router to said first 
memory controller, and via the first and second 
data routers to said second memory controller, at 
times when said DMA transfer is in the up direction 20 
and said DMA transfer will involve the first system 
resource bus and said DMA transfer will involve a 
system resource coupled to one of the system 
resource buses; 

wherein said upstream switching logic is configured 25 
to forward DMA data, from the second system 
resource bus, via the second data router to said 
second memory controller, and via the second and 
first data routers to said first memory controller, at 
times when said DMA transfer is in the up direction 30 
and said DMA transfer will involve the second 
system resource bus and said DMA transfer will 
involve a system resource coupled to one of the 
system resource buses; 

wherein said downstream switching logic is config- 35 
ured to fonward DMA data, from said first memory 
controller via the first data router, and from said 
second memory controller via the second and first 
data routers, to the first system resource bus, at 
times when said DMA transfer is in the down 40 
. direction and said DMA transfer will involve the first 
system resource bus and said DMA transfer will- 
involve a system resource coupled to one of the 
system resource buses; and 

wherein said downstream switching logic is config- 46 
ured to forward DMA data, from said second mem- ' 
ory controller via the second data router, and from 
said first memory controller via the first and second 
data routers, to the second system resource bus, at 
times when said DMA transfer is in the down so 
direction and said^ DMA transfer will involve the 
second system resource bus and said DMA trans- 
fer will Involve a system resource coupled to one of 
the system resource buses. 

28. A process in accordance with claim 24, wherein 55 
said switching logic includes downstream and up- . 
stream switching logic, and wherein the process 
comprises: 



configuring the downstream and upstream switch- 
ing logic in the first and second data routers, in 
response to transmission of said DMA start code, 
and in accordance with said DMA setup information 
decoded by the first and second data routers; 
wherein said upstream switching logic is configured 
to forward codes, from the first system resource 
bus, via the first data router to said first memory 
controller, and via the first and second data routers 
to said second memory controller, at times when 
said DMA transfer will involve the first system 
resource, bus and said DMA transfer will involve a 
system resource coupled to one of the system 
resource buses; 

wherein said upstream switching logic is configured 
to fonvard codes, from the second system resource 
bus, via the second data router to said second 
memory controller, and via the second and first 
data routers to said first memory controller, at 
times when said DMA transfer will involve the sec- 
ond system resource bus and said DMA transfer 
will Involve a system resource coupled to one of 
the system resource buses; 
wherein said upstream switching logic is configured 
to forward DMA data, from the first system re- 
source bus, via the first data router to said first 
memory controller, and via the first and second 
data routers to said second memory controller, at 
times when said DMA transfer Is in the up direction 
and said DMA transfer will involve the first system 
resource bus and said DMA transfer will involve a 
system resource coupled to one of the system 
resource buses; 

wherein said upstream switching logic is configured 
to forward DMA data, from the second system 
resource bus, via the second data router to said 
second memory controller, and via the second and 
first data routers to said first memory controller, at 
times when said DMA transfer is in the up direction 
and said DMA transfer will involve the second 
system resource bus and said DMA transfer will 
involve a system resource coupled to one of the 
system resource buses; 

wherein said downstream switching logic is config- 
ured to forward DMA data, from said first memory 
controller via the first data router, and from said 
second memory controller via the second and first 
data routers, to the first systeni resource bus. at 
times when said DMA transfer is in the down 
direction and said DMA transfer will involve the first 
system resource bus and said DMA transfer will 
involve a system resource coupled to one of the 
system resource buses; and 
wherein said downstream switching logic Is config- 
ured to fonrt^ard DMA data, from said second mem- 
ory controller via the second data router, and from 
said first memory controller via the first and second 
data routers, to the second system resource bus, at 
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times when said DMA transfer is in the down 
direction and said DMA transfer will involve the 
second system resource bus and said DMA trans- 
fer will involve a system resource coupled to one of 
the system resource buses. 
29. A process in accordance with claims 16, 17. 18, 
19, 22, or 23, wherein said switching logic includes 
downstream and upstream switching logic, and 
wherein the process comprises: 
configuring the downstream and upstream switch- 
ing logic in the first and second data routers, in 
response to transmission of said DMA start code, 
and in accordance with said DMA setup information 
decoded by the first and second data routers; 
wherein said upstream switching logic is configured 
to forward codes, from the first system resource 
bus. via the first data router to said first memory 
controller, and via the first and second data routers 
to said second memory controller, at times when 
said DMA transfer will involve the first system 
resource bus and said DMA transfer will Involve a 
system resource coupled to one of the system 
resource buses; 

wherein said upstream switching logic is configured 
to forward codes, from the second system resource 
bus, via the second data router to said second 
memory controller, and via the second and first 
data routers to said first memory controller, at 
times when said DMA transfer will involve the sec- 
ond system resource bus and said DMA transfer 
will involve a system resource coupled to one of 
the system resource buses; 
wherein said upstream switching logic is configured 
to forward DMA data, from the first system re- 
source bus, via the first data router to said first 
memory controller, and via the first and second 
data routers to said second memory controller, at 
times when said DMA transfer is In the up direction 
and said DMA transfer will involve the first system 
resource bus and said DMA transfer will involve a 
system resource coupled to one of the system 
resource buses: 

wherein said upstream switching logic is configured 
to forward DMA data, from the second system 
resource bus, via the second data router to said 
second memory controller, and via the second and 
first data routers to said first memory controller, at 
times when said DMA transfer is in the up direction 
and said DMA transfer will involve the second 
system resource bus and said DMA transfer will 
involve a system resource coupled to one of the 
system resource buses; 

wherein said downstream switching logic is config- 
ured to forward DMA data, from said first memory 
controller via the first data router, and from said 
second memory controller via the second and first 
data routers, to the first system resource bus, at 
times when said DMA transfer is in the down 



direction and said DMA transfer will involve the first 
system resource bus and said DMA transfer will 
involve a system resource coupled to one of the 
system resource buses; and 

5 wherein said downstream switching logic is config- 
ured to forward DMA data, from said second mem- 
ory controller via the second data router, and from 
said first memory controller via the first and second 
data routers, to the second system resource bus. at 

10 times when said DMA transfer is in the down 
direction and said DMA transfer will involve the 
second system resource bus and said DMA trans- ^ 
fer will Involve a system resource coupled to one of 
the system resource buses. 

75 30. A process for transferring data via DMA be- 
tween components in a computer system, wherein 
the computer system includes a mernory controller, 
. a first system resource bus coupled to a system 
resource, and a data router, wherein the data router 

20 includes switching logic for coupling the memory 
controller to the first system' resource bus. and 
wherein the process comprises the following steps: 
transmitting DMA setup information designating a 
setup write transaction, from the memory controller, 

26 to the system resource, via the data router and via 
the first system resource bus coupled to said sys- 
tem resource, wherein said DMA setup information 
corresponds to a DMA transfer, and indicates a 
direction of the DMA transfer as either an up direc- 

30 tion from said system resource to said memory 
controller or a down direction^ from said memory 
•controller to said system resource, indicates wheth-' 
er said DMA transfer will Involve said first system 
resource bus. and Indicates whether said DMA 

35 transfer will Involve a system resource coupled to 
said first system resource bus; 
decoding said DMA setup information, in the data 
router, during said setup write transaction, to deter- 
mine the direction of said DMA transfer, to deter- 

40 rnine whether said DMA transfer will involve said 
'first system resource bus. and to determine wheth- 
er said DMA transfer will involve a system resource 
coupled to said first system resource bus; 
transmitting a start DMA code, from said memory 

46 controller, to said system resource, via the data 
router and via said first system resource bus coup- 
led to said system resource; 
configuring the switching logic in said data router, 
in response to transmission of said DMA start 

50 code, and in accordance with said DMA setup 
information decoded by said data router: 
wherein said switching logic is configured to for- 
ward DMA data, from said first system resource 
bus to said memory controller, at times when said 

55 DMA transfer Is in the up direction and said DMA 
transfer will involve said first system resource bus 
and said DMA transfer will involve a system re- 
source coupled to said first system resource bus; 
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and 

wherein said switching logic is configured to for- 
ward DMA data, from said memory controller, to 
said first system resource bus, at times when said 
DMA transfer is in the down direction and said 5 
DMA transfer will involve said first system resource 
bus and said DMA transfer will involve a system 
resource coupled to said first system resource bus; 
transmitting an acknowledge code» from said sys- 
tem resource, to said memory -controller, via said 10 
first system resource bus coupled to said system 
resource, and via the switching logic in said data 
router, in response to transmission of said DMA 
start code; 

transmitting DMA data, after transmission of said 75 
acknowledge code, at times when said DMA trans- 
fer is in the up direction, from said system re- 
source, to said memory controller, via said first 
system resource bus coupled to said system re- 
source, and via the switching logic in said data 20 
router; 

transmitting DMA data, after transmission of said 
acknowledge code, at times when said DMA trans- 
fer is in the down direction, from said memory 
controller, to said system resource, via the switch- 25 
ing logic in said data router, and via said first 
system resource bus coupled to said system re- 
source; and 

transmitting a done code, after transmission of 
DMA data, from said system resource, to said 30 
memory controller, via said first system resource 
bus coupled to said system resource, and via the " 
switching logic in said data router, to indicate suc- 
cessful DMA data transmission. 

31. A process for transfemng data via DMA be- 3S 
tween components in a computer system; wherein 
the computer system includes a first primary, mem- 
ory controller, a first mirror memory controller, a 
second primary memory controller, a second mirror 
memory controller, a first primary system resource - 40 
bus and a first mirror system resource bus coupled 
to a first system resource, a second primary sys- 
tem resource bus and a second mirror system 
resource bus coupled to a second system re- 
source, a first primary data router coupled to the 45 
first primary memory controller and to the first 
primary system resource bus, a first mirror data 
router coupled to the first min-or memory controller 
and to the first mirror system resource bus, a 
second primary data router coupled to the second 50 
primary memory controller and to the second pri- 
mary system resource bus. a second mirror data 
router coupled to the second min-or memory con- 
troller and to the second mirror system resource 
bus, wherein each of the data routers includes 55 
switching logic for coupling the memory controllers 
to the system resource buses, wherein the first 
data routers are. coupled to the second system 



resource buses via the switching logic in the sec- 
ond data routers, and the second data routers are 
coupled to the first system resource buses via the 
switching logic in the first data routers, and wherein 
the process comprises the following steps: 
transmitting the same DMA setup Information des- 
ignating a setup write transaction, from each of the 
memory controllers, to one of the system re- 
sources, via the first and second data routers and 
via the. pair of system resource buses coupled to 
said system resource, wherein said DMA setup 
information corresponds to a DMA transfer, and 
indicates a direction of the DMA transfer as either 
an up direction from said system resource to said 
memory controllers or a down direction from said 
memory controllers to said system resource, in- 
dicates which pair of system resource buses the 
DMA transfer will Involve, and indicates whether 
said DMA transfer will involve a system resource 
coupled to one pair of the system resource buses; 
decoding said DMA setup information in the first 
and second data routers during said setup write 
transaction, to determine the direction of said DMA 
transfer, to determine which pair of system re- 
source buses said DMA transfer will involve, and to 
determine whether said DMA transfer will involve a 
system resource coupled to one pair of the system 
resource buses; 

transmitting a start DMA code, from each of said 
memory controllers, to said system resource, via 
the first and second data routers, and via the pair 
of system resource buses coupled to said system 

resource; 

configuring the switching logic in the first and sec- 
ond data routers, in response to transmission of 
said DMA start code, and in accordance with said 
DMA setup Information decoded by the first and 
second data routers; 

wherein said switching logic is configured to for- 
ward DMA data, from the pair of first system re- 
source buses, via the switching logic in the pair of 
first data routers to the pair of first memory control- 
lers, and via the switching logic in the pairs of first 
and second data routers to the pair of second 
memory controllers, at times when said DMA trans- 
fer is In the up direcfion and said DMA transfer will 
involve the pair of first system resource buses and 
said DMA transfer will involve a system resource 
coupled to one pair of the system resource buses; 
wherein said switching logic is configured to for- 
ward DMA data, from the pair of second system 
resource buses, via the switching logic in the pair 
of second data routers to the pair of second mem- 
ory controllers, and via the switching logic in the 
pairs of second and first data routers to the pair of 
first memory controllers, at fimes when said DMA 
transfer is in the up direction and said DMA trans- 
fer will involve the pair of second system resource 
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buses and said DMA transfer will involve a system 
resource coupled to one pair of the system re- 
source buses; 

wherein said switching logic is configured to for- 
ward DMA data, from one of the first memory s 
controllers via the switching logic in one of the first 
data routers, and from one of the second memory 
controllers via the switching logic in one of the 
second data routers and the other first data router, 
to the pair of first system resource buses, at times to 
when said DMA transfer is in the down direction 
and said DMA transfer wilt involve the pair of first 
system resource buses and said DMA transfer will 
involve a system resource coupled to one pair of 
the system resource buses; and 75 • 

wherein said switching logic Is configured to for- 
ward DMA data, from one of the first memory 
controllers via the switching logic In one of the first 
data routers and one of the second data routers, 
and from one of the second memory controllers via 20 
the switching logic in the other second data router, 
to the pair of second system resource buses, at 
times when said DMA transfer is In the down ' 
direction and said DMA transfer will involve the pair 
of second system resource buses and said DMA 25 
transfer will involve a system resource coupled to 
one pair of the system resource buses; 
transmitting an acknowledge code, from said sys- 
tem resource, to each of said memory controllers, 
via the pair of system resource buses coupled to 30 
said system resource, and via the switching logic in 
the first and second data routers, in response to 
transmission of said DMA start code; 
transmitting DMA data, after transmission of said 
acknowledge code, at times when said DMA trans- 35 
fer is in the up direction, from said system re- 
source, to each of said memory controllers, via the 
pair of system resource buses coupled to said 
system resource, and via the switching logic in the 
first and second data routers; 40 
transmitting DMA data, after transmission of said 
acknowledge code, at times when said DMA trans- 
fer is In the down direction, from one of the first 
memory controllers and from one of the second 
memory controllers, to said system resource, via 45 
the switching logic in the first and second data 
. routers, and via the pair of system resource buses 
coupled to said system resource; and 
transmitting a done code, after transmission of 
DMA data, from said system resource, to each of so 
said memory controllers, via the pair of system 
resource buses coupled to said system resource, 
and via the switching logic In the first and second 
data routers, to indicate successful DMA data 
transmission. 55 
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